Finding errors without a simulator (Oshonsoft)

[camerart]

Hi,

I'm checking a routine in my CODE that checks serial input sentences.
A sentence could look like this: $ROBI,111,1,11,W or
this: $GNRMC,111111.00,A,3723.02837,N,00150.39853,W,0.820,188.36,110706,,,A*74
So all of them start with a $ then some CHARs
If an error sentence was used, this could cause a catastrophe.

The routine I'm checking looks for the $ and the first few CHARs and compares them with the $CHARs expected.
I have a mate who is helping me, and he doesn't use the SIM, where I have to.
Using the SIM, I sent a long string of sentences, all with incrementing DATA but some with CHAR errors, and found the error in the routine, and corrected it, so it rejects the error sentences.

The error I found was a sort of negative one, where, as it would be rare to receive an error sentence, they would be passed.
My question is, how does a none SIM user check CODE like this?
Camerart.

 

[rjenkinsgb]

Use the proper checksum algorithm?

NMEA Checksums Explained

I recently spent a ton of time debugging my code to find a problem that was ultimately due to an incorrect implementation of the checksum calculation used for NMEA GPS messages. I'm hoping that by sharing what I learned through the process that someone else might avoid similar issues. NMEA...

forum.arduino.cc

Looking through that:

It looks like you start with a zero checksum byte at the start of each sentence, then XOR each character after the $ with that byte, until you get to the * (but don't XOR that).

The checksum byte should, if there are no errors, then equal the two characters after the * when those are translated from hexadecimal (or checksum byte is converted to hex).


For general debugging, add lost of print statements to show intermediate results??

 

[camerart]

Use the proper checksum algorithm?

NMEA Checksums Explained

I recently spent a ton of time debugging my code to find a problem that was ultimately due to an incorrect implementation of the checksum calculation used for NMEA GPS messages. I'm hoping that by sharing what I learned through the process that someone else might avoid similar issues. NMEA...

forum.arduino.cc

Looking through that:

It looks like you start with a zero checksum byte at the start of each sentence, then XOR each character after the $ with that byte, until you get to the * (but don't XOR that).

The checksum byte should, if there are no errors, then equal the two characters after the * when those are translated from hexadecimal (or checksum byte is converted to hex).


For general debugging, add lost of print statements to show intermediate results??

Hi R,
Note, that the 2x different types of sentence start with $ and have a W, which is what the CODE stops at, so no * or the end of sentence checksum.
I've never used checksums, and the data changes but the start CHARS don't, does your reply cover this?
C

 

[rjenkinsgb]

ROBI is not a valid NMEA sentence, as far as I can see?? Do you have any info on what it relates to?

All sentences are supposed to have the *checksum at the end, according to the NMEA standard.
(See the "Coding conventions" section in this: )

NMEA Revealed

A programmer's guide to NMEA0183, the GPS protocol reporting standard

gpsd.gitlab.io

A simple work-around would be to only do the checksum if the first character after the $ was G

 

[camerart]

ROBI is not a valid NMEA sentence, as far as I can see?? Do you have any info on what it relates to?

All sentences are supposed to have the *checksum at the end, according to the NMEA standard.
(See the "Coding conventions" section in this: )

NMEA Revealed

A programmer's guide to NMEA0183, the GPS protocol reporting standard

gpsd.gitlab.io

A simple work-around would be to only do the checksum if the first character after the $ was G

Hi R,
As you know the GPS NMEA sentences have a checksum, but I don't use it, as I only need DATA between $ and in this instance W. The ROBI sentence was designed by me to imitate the NMEA format, starting with $ and ending with W.

The CODE where I found the error, checks the fixed CHARs in each sentnece, in a similar way to your suggestion of G.
C.

 

[Nigel Goodwin]

Hi,

I'm checking a routine in my CODE that checks serial input sentences.
A sentence could look like this: $ROBI,111,1,11,W or
this: $GNRMC,111111.00,A,3723.02837,N,00150.39853,W,0.820,188.36,110706,,,A*74
So all of them start with a $ then some CHARs
If an error sentence was used, this could cause a catastrophe.

The routine I'm checking looks for the $ and the first few CHARs and compares them with the $CHARs expected.
I have a mate who is helping me, and he doesn't use the SIM, where I have to.
Using the SIM, I sent a long string of sentences, all with incrementing DATA but some with CHAR errors, and found the error in the routine, and corrected it, so it rejects the error sentences.

The error I found was a sort of negative one, where, as it would be rare to receive an error sentence, they would be passed.
My question is, how does a none SIM user check CODE like this?

As with anything else, you simply use I/O pins as debug pins - either toggle them high or low at specific points, or under specific conditions, you can check them with either an LED on the pin, or with a scope, depending what you're doing - I often toggle a pin before and after a routine, so you can measure how long the routine takes using a scope (doing this at the input and output of an ISR shows you how much time your ISR is actually taking). I usually use a pin as serial I/O as well, either hardware (if available) or software, and use #ifdef/#endif statements out the serial port if required, like this:

#ifdef Debug_Values
    printf("DataValue Hex = 0x0%x", DataValue);
    printf(" - Command Hex = 0x0%x", CommandValue);
    printf(" - Device Hex = 0x0%x", DeviceValue);
    if (irparams.PulseNum==21)                                      // check if 20 bit code?
        printf(" - Extra Hex = 0x0%x", DataValue>>12);
    printf("\n\r");
#endif

This particular clip is from an IR remote receiver routine - you simply #define Debug_Values at the beginning of the program, and this bebug section is added.

I often leave similar code in place in some parts of the final products, to make fault finding in the future mush easier - again, depending on the exact circumstances.

I've never used simulators, but I come from a time before them - and the first PIC simulators (under DOS) were REALLY horrible

 

[camerart]

As with anything else, you simply use I/O pins as debug pins - either toggle them high or low at specific points, or under specific conditions, you can check them with either an LED on the pin, or with a scope, depending what you're doing - I often toggle a pin before and after a routine, so you can measure how long the routine takes using a scope (doing this at the input and output of an ISR shows you how much time your ISR is actually taking). I usually use a pin as serial I/O as well, either hardware (if available) or software, and use #ifdef/#endif statements out the serial port if required, like this:

#ifdef Debug_Values
    printf("DataValue Hex = 0x0%x", DataValue);
    printf(" - Command Hex = 0x0%x", CommandValue);
    printf(" - Device Hex = 0x0%x", DeviceValue);
    if (irparams.PulseNum==21)                                      // check if 20 bit code?
        printf(" - Extra Hex = 0x0%x", DataValue>>12);
    printf("\n\r");
#endif

This particular clip is from an IR remote receiver routine - you simply #define Debug_Values at the beginning of the program, and this bebug section is added.

I often leave similar code in place in some parts of the final products, to make fault finding in the future mush easier - again, depending on the exact circumstances.

I've never used simulators, but I come from a time before them - and the first PIC simulators (under DOS) were REALLY horrible

Hi N,
I see you've had lots of practice to know an alternative to SIMs.
As you're CODE isn't in BASIC, I can only guess how it works

With the simulator, I can feed a string of correct and incorrect sentences, toggle pins, add break points, step through speed up and slow down the sim. I'm no programmer, but I found the error before my mate! (He doesn't spend as much time as me though )
C

 

[rjenkinsgb]

As you know the GPS NMEA sentences have a checksum, but I don't use it, as I only need DATA between $ and in this instance W. The ROBI sentence was designed by me to imitate the NMEA format, starting with $ and ending with W.

Without the checksum, there is no way of knowing if digits have been incorrectly received - and doing the checksum is probably quicker than many comparisons to check letters are in the correct places?

 

[camerart]

Without the checksum, there is no way of knowing if digits have been incorrectly received - and doing the checksum is probably quicker than many comparisons to check letters are in the correct places?

Hi R,
Ok, is it possible to add a checksum to a GPS NMEA sentence, $ to W?
C

 

[Nigel Goodwin]

Hi R,
Ok, is it possible to add a checksum to a GPS NMEA sentence, $ to W?
C

You don't 'add' a checksum, it's sent out by the transmission, and you compare it to the checksum you yourself calculate from that transmission. If the checksums don't match, you reject the transmission.

But, as far as I'm aware, the NMEA strings are really long, so you could be struggling for memory to process them?.

 

[DogFlu66]

I have had to work with NMEA frames several times but I have not had to process them.

'Function for validating nmea frames.
'Returns 1 If the size Checksum is correct.
'Input value a string variable with NMEA format.
Function _NMEA_Checksum(_string As String) As Byte

Symbol _Return = _NMEA_Checksum
Dim _Ck_h As Byte
Dim _Ck_l As Byte
Dim Checksum As Byte
Dim x As Byte
Checksum = 0
x = 0
_Return = 0
'Calculate the checksum and locate the checksum of the frame.
While _string(x) <> "*"
x++
If _string(x) <> "*" Then Checksum = Checksum Xor _string(x)
If _string(x) = 0 Then Exit 'Reaches the end of the string, exits the function.
Wend
'Extract nibbles from checksum
_Ck_h = _getnibblehb(Checksum)
_Ck_l = _getnibblelb(Checksum)
'Passes it to ascii value
If _Ck_h < 10 Then
_Ck_h = _Ck_h + 48
Else
_Ck_h = _Ck_h + 55
Endif
'Passes it to ascii value
If _Ck_l < 10 Then
_Ck_l = _Ck_l + 48
Else
_Ck_l = _Ck_l + 55
Endif
'If the checksum of the string is equal To the calculated one, it returns 1.
If _string(x + 1) = _Ck_h And _string(x + 2) = _Ck_l Then _Return = 1

End Function

 

[camerart]

You don't 'add' a checksum, it's sent out by the transmission, and you compare it to the checksum you yourself calculate from that transmission. If the checksums don't match, you reject the transmission.

But, as far as I'm aware, the NMEA strings are really long, so you could be struggling for memory to process them?.

Hi N,
This is why I stop at W, so I don't put the whole string in the buffer, so no checksum.
The other type of string: $ROBI,111,1,11,W is copying the style of $ to W, which I send backwards and forwards on my project, so again no checksum.
C

 

[camerart]

I have had to work with NMEA frames several times but I have not had to process them.

'Function for validating nmea frames.
'Returns 1 If the size Checksum is correct.
'Input value a string variable with NMEA format.
Function _NMEA_Checksum(_string As String) As Byte

Symbol _Return = _NMEA_Checksum
Dim _Ck_h As Byte
Dim _Ck_l As Byte
Dim Checksum As Byte
Dim x As Byte
Checksum = 0
x = 0
_Return = 0
'Calculate the checksum and locate the checksum of the frame.
While _string(x) <> "*"
x++
If _string(x) <> "*" Then Checksum = Checksum Xor _string(x)
If _string(x) = 0 Then Exit 'Reaches the end of the string, exits the function.
Wend
'Extract nibbles from checksum
_Ck_h = _getnibblehb(Checksum)
_Ck_l = _getnibblelb(Checksum)
'Passes it to ascii value
If _Ck_h < 10 Then
_Ck_h = _Ck_h + 48
Else
_Ck_h = _Ck_h + 55
Endif
'Passes it to ascii value
If _Ck_l < 10 Then
_Ck_l = _Ck_l + 48
Else
_Ck_l = _Ck_l + 55
Endif
'If the checksum of the string is equal To the calculated one, it returns 1.
If _string(x + 1) = _Ck_h And _string(x + 2) = _Ck_l Then _Return = 1

End Function

View attachment 143691

Hi D,
Thanks for the CODE, I'll keep it, as my mate may add checksums to the shorter sentence style ($ROBI,111,1,11,W), if appropriate, and this will come in handy then.
C

 

[sagor1]

You don’t have to store the entire message, just up to the W, but your receiving routine can still checksum the entire string to make sure it is valid. Your UART is going to get all of the NEMA string anyway, including the sent checksum, so might as well check it and set a validity flag.
So, capture what you need, but check the entire string anyway, it all goes through your receive uart code anyway.

 

[rjenkinsgb]

This is why I stop at W, so I don't put the whole string in the buffer, so no checksum.

You don’t have to store the entire message, just up to the W, but your receiving routine can still checksum the entire string to make sure it is valid.

Exactly.

As you receive characters, set the checksum byte to zero when you see the initial $, then XOR every character with that byte until you get to the *
Then do the checksum hex conversion and comparison.

You can store as little or as much of the incoming string as you wish, it's just the XOR part you need to do before storing or discarding a character.

 

[camerart]

Exactly.

As you receive characters, set the checksum byte to zero when you see the initial $, then XOR every character with that byte until you get to the *
Then do the checksum hex conversion and comparison.

You can store as little or as much of the incoming string as you wish, it's just the XOR part you need to do before storing or discarding a character.

Hi R,
Ok, I'll try to do this, on the NMEA sentences, but I suppose the $ROBI,111,1,11,W type messages need to have the checksum added when they are sent?
C

 

[DogFlu66]

Adds checksum:

'Adds string termination and the two checksum digits to the input string.
'Input value a string variable with NMEA format.
Proc _NMEA_Checksum_Add(ByRef _string As String)

Dim _Ck_h As Byte
Dim _Ck_l As Byte
Dim Checksum As Byte
Dim x As Byte
Checksum = 0
x = 0
If _string(0) = 0 Then Exit 'Empty string, exit the function.
'Calculate the checksum and locate the checksum of the frame.
While _string(x) <> 0
x++
If _string(x) <> 0 Then Checksum = Checksum Xor _string(x)
If _string(x)= "*" Then Exit 'Has checksum, exit function.
Wend
'Extract nibbles from checksum
_Ck_h = ShiftRight(Checksum ,4)
_Ck_l = Checksum And %00001111
'Passes it to ascii value
If _Ck_h < 10 Then
_Ck_h = _Ck_h + 48
Else
_Ck_h = _Ck_h + 55
Endif
'Passes it to ascii value
If _Ck_l < 10 Then
_Ck_l = _Ck_l + 48
Else
_Ck_l = _Ck_l + 55
Endif
'Adds NMEA data character termination.
_string(x) = "*"
'Adds the two Checksum digits.
_string(x + 1) = _Ck_h
_string(x + 2) = _Ck_l
_string(x + 3) = 0 'Updates the end of string indicator.

End Proc

I'm thinking it would be good to open a new thread with examples.

 

[rjenkinsgb]

(ignore)

 

[camerart]

Adds checksum:

'Adds string termination and the two checksum digits to the input string.
'Input value a string variable with NMEA format.
Proc _NMEA_Checksum_Add(ByRef _string As String)

Dim _Ck_h As Byte
Dim _Ck_l As Byte
Dim Checksum As Byte
Dim x As Byte
Checksum = 0
x = 0
If _string(0) = 0 Then Exit 'Empty string, exit the function.
'Calculate the checksum and locate the checksum of the frame.
While _string(x) <> 0
x++
If _string(x) <> 0 Then Checksum = Checksum Xor _string(x)
If _string(x)= "*" Then Exit 'Has checksum, exit function.
Wend
'Extract nibbles from checksum
_Ck_h = ShiftRight(Checksum ,4)
_Ck_l = Checksum And %00001111
'Passes it to ascii value
If _Ck_h < 10 Then
_Ck_h = _Ck_h + 48
Else
_Ck_h = _Ck_h + 55
Endif
'Passes it to ascii value
If _Ck_l < 10 Then
_Ck_l = _Ck_l + 48
Else
_Ck_l = _Ck_l + 55
Endif
'Adds NMEA data character termination.
_string(x) = "*"
'Adds the two Checksum digits.
_string(x + 1) = _Ck_h
_string(x + 2) = _Ck_l
_string(x + 3) = 0 'Updates the end of string indicator.

End Proc

View attachment 143708

I'm thinking it would be good to open a new thread with examples.

Hi D,
Good, I'll put it with the other one, and play with them later, thanks.
When I 'play', I'll perhaps start another thread!
C

 

[camerart]

Hi,
I think the answer to my initial question, is that good programmers see CODE in a more black and white way than I do.
When going through CODE, I get side tracked sometimes, and loose my place, and have difficulty proof reading backwards and forwards, especially with similar named variables.
I think a good coder would be able to switch on and off the bits etc as they read through.
Not a very clear explanation, but I bet it's something like that.
C