1. Welcome to our site! Electro Tech is an online community (with over 170,000 members) who enjoy talking about and building electronic circuits, projects and gadgets. To participate you need to register. Registration is free. Click here to register now.
    Dismiss Notice

Single Data Type Rounding Error

Discussion in 'Oshonsoft' started by Mity Eltu, Dec 19, 2015.

  1. Mity Eltu

    Mity Eltu Member

    Joined:
    Apr 1, 2014
    Messages:
    49
    Likes:
    0
    Location:
    Tennessee, USA
    I am using a single data type variable to take a running average of four numbers. I made this really simple so I could verify that it is doing what it is supposed to do, but I find it doesn't do what I think it should.

    Here's the code:
    Code (text):
    '18F4520
    'LCD Configuration - 4-bit lower
    Define LCD_LINES = 2
    Define LCD_CHARS = 16
    Define LCD_BITS = 4
    Define LCD_DREG = PORTD
    Define LCD_DBIT = 4
    Define LCD_RSREG = PORTD
    Define LCD_RSBIT = 0
    Define LCD_RWREG = PORTD
    Define LCD_RWBIT = 1
    Define LCD_EREG = PORTD
    Define LCD_EBIT = 2

    Define CLOCK_FREQUENCY = 8
    Define SINGLE_DECIMAL_PLACES = 6
    Define SIMULATION_WAITMS_VALUE = 10

    'Global Variables
    Dim flags As Byte
    Dim vals(4) As Single
    Dim i As Byte
    Dim ii As Byte
    Dim avg As Single
    Dim num As Single
    Dim cnt As Single

    main:
        OSCCON = 0x72  'Internal 8MHz osc
        Gosub setup
       
        Lcdinit 0
       
        Lcdout "Ready"
        WaitMs 1000
        Lcdcmdout LcdClear
        Lcdout "Average = "
       
       
    loop:
        'running average:
        'array of 4 numbers, [0-3]
        'first four are loaded with index 0-3, summed
        'and divided by 4
        'then  start replacing each number 0-3 where
        'index 0 holds the oldest, and hence, the
        'smallest number
       
        If flags.0 = 0 Then  'new number coming in NOTE: I made this so it would just run instead of waiting for the timer interrupt
            flags.0 = 0
            num = cnt * 1.10000
            vals(i) = num
            avg = 0
            For ii = 0 To 3
                avg = avg + vals(ii)
            Next ii
            avg = avg / 4.00000
            Lcdcmdout LcdLine2Clear
            Lcdout #avg
            i = i + 1
            If i = 4 Then i = 0
            cnt = cnt + 1.00000
        Endif

        Goto loop
    End                                              


    setup:
        INTCON = 0xd0  'int0 en
        INTCON2 = 0xc0  'pull ups disabled, int0 on rising egde
        PIE2.1 = 1  'tmr3 int enabled
        RCON.7 = 1  'priority ints enabled
        IPR2.1 = 1  'tmr3 int hi priority
       
        CCP1CON = 0x00  'cap / comp off
        CCP2CON = 0x00
       
        SSPCON1 = 0x21  'spi on, fosc/16, clk idel state low
        SSPSTAT = 0x40  'sample in middle, cke=1 - xfer on clk high to low
       
        ADCON0 = 0x00  'A/D off
        ADCON1 = 0x0f  'all digital
       
        CMCON = 0x07  'comp off, digital io
        CVRCON = 0x00
           
        HLVDCON = 0x00  'Low volt detect off
       
        TRISA = 0x00
        TRISB = 0x01
        TRISC = 0x00
        TRISD = 0x00
        TRISE = 0x00
       
        LATA = 0x00
        LATB = 0x00
        LATC = 0x00
        LATD = 0x00
        LATE = 0x01  'porte0 set for MAX6675 chip select

        'set timer3 for interrupt for logging new number
        'for demo use timer0 and 256 prescale
        T3CON = 0x30  'tmr3 1:8 (262mS) prescale, off
       
        flags = 0x00
        i = 0  'array index
        cnt = 1
    Return                                          


    On High Interrupt
        If INTCON.1 = 1 Then  'int0
            INTCON.1 = 0
            flags.0 = 1  'set val to be added
        Endif
       
        'if tmier ovf, set flag1 to get new number
        If PIR2.1 = 1 Then  'tmr3
            PIR2.1 = 0
            flags.0 = 1
        Endif
    Resume                                            
    And here is a list of the averages it gives:
    0.275000 - correct
    0.824999 - incorrect. should be 0.285
    1.649999 - incorrect. should be 1.65
    2.750000 - correct. weird
    3.849999 - incorrect. should be 3.85
    the rest are incorrect from here out following the same behavior of x.xx9999.

    Is there some setting I need to change or is this just a function of the single precision data type? I don't believe there is a round command in the basic compiler, so is there a simple way to fix this?
     
  2. ericgibbs

    ericgibbs Well-Known Member Most Helpful Member

    Joined:
    Jan 4, 2007
    Messages:
    21,180
    Likes:
    644
    Location:
    Ex Yorks' Hants UK
    hi Mity,
    This is your program on my PC, I have used the IDE Uart to show a printed record of the 'sums'.
    I have noticed these rounding errors when calculating GPS data angles etc.

    Eric.
    BTW: I have spent ages on the SD card trying to get some of the commands working in a reliable way, looking forward to seeing your latest SD prog.
     

    Attached Files:

    • A36.gif
      A36.gif
      File size:
      16.5 KB
      Views:
      93
  3. sagor1

    sagor1 Member

    Joined:
    Dec 11, 2014
    Messages:
    52
    Likes:
    2
    Mity, what you are seeing is the "single" precision accuracy, which is 6 to 7 digits (mostly 7). That is what causes the "9999" types of answers (I assume the 0.824999 you really mean 0.284999). This is a common error in any single float, it cannot guarantee the last digit, it could be +/- 1 digit. You can trim this with some form or "rounding", but I don't see that function in this compiler...
    Using float (single) variables with 7 digit precision will always have +/- 1 digit error for the last digit. IEEE 754 standard clearly states "This gives from 6 to 9 significant digits", meaning you can only be 100% sure of the first 6.
     
  4. dave

    Dave New Member

    Joined:
    Jan 12, 1997
    Messages:
    -
    Likes:
    0


     
  5. Ian Rogers

    Ian Rogers Super Moderator Most Helpful Member

    Joined:
    Mar 28, 2011
    Messages:
    9,157
    Likes:
    907
    Location:
    Rochdale UK

    I did a complex breakdown of floating point format some time ago!!! It's a very weird way of representing a number but there isn't anything better....

    Oshonsoft does have the DECIMAL_PLACES directive but I don't know whether it rounds up or down ( I would have thought down )

    In a single there will be 24 mantissa bits and unless the decimal fraction is weighted to a "achievable" bit there will always be re-occurring 3's, 6's and 9's.... In fact any number that doesn't divide into 10 so unless the last digit resolves to a 1,2 or 5... trouble... The though behind it.

    2.499999 is only 0.000001 in error so who gives a jot!!

    The mantissa fractions divide down bit by bit, each bit is weighted so:-

    0.5, 0.25, 0.125, 0.0625, 0.03125... etc... so there WILL be numbers that will be out of reach
     
  6. Mity Eltu

    Mity Eltu Member

    Joined:
    Apr 1, 2014
    Messages:
    49
    Likes:
    0
    Location:
    Tennessee, USA
    I understand, from the big picture kind of perspective, how this single precision stuff works, but these numbers are all well within the 7 significant digit representation. The compiler, for whatever reason, rounds everything down, so that 0.284999 comes to be 0.284 when the decimal places is set to 3. That makes no sense to me whatsoever, but it is what it is.

    I guess I'll have to deal with that or switch back to sourceboost c++ and try to bang out the sd card interface.

    I think I'd rather have mu gums scraped.

    Thanks.
     
  7. Ian Rogers

    Ian Rogers Super Moderator Most Helpful Member

    Joined:
    Mar 28, 2011
    Messages:
    9,157
    Likes:
    907
    Location:
    Rochdale UK
    The seven decimal place can be hard to achieve with a single precision... It has nothing to do with Oshonsoft, but some compilers will cater for the tiny discrepancy... When the number overflows it should technically be rounded up.. Online calculators do... But Oshonsoft is a cheap compiler..
     
  8. camerart

    camerart Active Member

    Joined:
    Jun 12, 2008
    Messages:
    1,315
    Likes:
    11
    Location:
    Dorset UK.
    Hi,

    I'm also having the same problem.

    I think I read somewhere that only '1' single can be used in each program. Is this correct?

    For what I need: There are 4 numbers with digits after the decimal place. I don't need complete accuracy, so if I multiply them up till they are whole numbers, i,e, (For 100 multiplied by 'say' 0.001) will 100X1000 solve my problem?

    Camerart.
     
  9. Mity Eltu

    Mity Eltu Member

    Joined:
    Apr 1, 2014
    Messages:
    49
    Likes:
    0
    Location:
    Tennessee, USA
    No, you can have more than one single in a program. The number of singles might be limited due to their size (32 bits), but as long as there is space, you can have as many as you want.

    If you are going to scale them up by 1000, you will need to make sure that the data type you have selected for the answer is large enough to handle the largest POSSIBLE result. For instance, lets say you decide t use the following:

    Dim num1 As Single
    Dim num2 As Word

    num1 = 0.001
    num2 = num1 * 1000

    This will work just fine, but if num1 is ever larger than 65.536, your result will be incorrect as a word data type cannot hold a number larger than 65536.

    Hope that helps.
     
  10. jjw

    jjw Member

    Joined:
    Apr 16, 2012
    Messages:
    258
    Likes:
    15
    Location:
    Helsinki, Finland
    The compiler does not round down, but truncates the number, when displayed.
    It can be handled, for example to display three decimal places, add 0.0005 to the number before displaying it.
     
  11. camerart

    camerart Active Member

    Joined:
    Jun 12, 2008
    Messages:
    1,315
    Likes:
    11
    Location:
    Dorset UK.
    Hi M,

    Yes it helps, thanks.

    If the number reaches larger than the POSSIBLE result, you say it will be incorrect. In simple terms could give two numbers that are the same, result in not being the same?

    C.
     
  12. jjw

    jjw Member

    Joined:
    Apr 16, 2012
    Messages:
    258
    Likes:
    15
    Location:
    Helsinki, Finland
    If you don't need decimals, you can simply cast the floating point number to an integer.
    dim a as single
    dim b as word
    b=a ' b will be the integer part of a

    What is the range of your variables? If > 65535 then they must be long variables.
     
  13. camerart

    camerart Active Member

    Joined:
    Jun 12, 2008
    Messages:
    1,315
    Likes:
    11
    Location:
    Dorset UK.
    Hi J,

    I've been trying to use decimals (SINGLE), but they never rounded up properly, what ever I tried. I made a test program using LONGs and no SINGLES. multiplying the decimal points out of the variables, then dividing them down to what they should be, and this appears to be doing what I want.

    EDITED.

    Thanks, C.
     
    Last edited: Feb 7, 2016

Share This Page