Go Down

Topic: Serial Input Basics: example 5, question (Read 2224 times) previous topic - next topic

Robin2

EDIT: the format currently tested is with A. hence: <47.50,12.50,6.50,5,10,A.>
Then it seems that you should check that the final character is a full-stop.

IF the final character is a full-stop AND the program crashes sometimes then I don't know what is causing it. The ESP8266 is a lot more complex than an Uno because it has all that WiFi code that needs to run at regular intervals. Is there any chance that something in your code (maybe nothing to do with parsing) is interfering with the WiFi stuff and the crash is actually happening somewhere else?

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

brice3010

Then it seems that you should check that the final character is a full-stop.

IF the final character is a full-stop AND the program crashes sometimes then I don't know what is causing it. The ESP8266 is a lot more complex than an Uno because it has all that WiFi code that needs to run at regular intervals. Is there any chance that something in your code (maybe nothing to do with parsing) is interfering with the WiFi stuff and the crash is actually happening somewhere else?

...R
I also thought about that until this stripped down version of my program (ref. to your helpfull suggestion) was tested before cattledog's suggestion of A. ; but then too I got these crashes. And I use the yield; command to allow these background processes to perform. After implementing cattledog's A. suggestion there was a major improvement.

I have an ESP32 where this program compiles correctly, but the hardware is not set up yet for the HC12 radio connection; after that is ready, and after testing all of the current suggestions, that is going to be plan B.

brice3010

Just a few minutes into testing the results are worse than before:
"Missing ." shows up every second or third or fourth parse from identifier B, however the sent message does contain B. so in the sent message the output is correct.
But every time after "Missing ." shows up the ESP8266 crashes.
Now: revert back to the previous setup, without the "Missing ." check and see what happens the next 24h. To make sure nothing else is at play.


Robin2

Just a few minutes into testing the results are worse than before:
"Missing ." shows up every second or third or fourth parse from identifier B, however the sent message does contain B. so in the sent message the output is correct.
The results may be worse but the cause of the problem may be in sight.

Post the program.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

brice3010

#34
Jan 18, 2019, 07:33 am Last Edit: Jan 18, 2019, 07:35 am by brice3010
The results may be worse but the cause of the problem may be in sight.

Post the program.

...R
Here is the program including the latest terminator verification/rectification as suggested by cattledog.

Robin2

This code
Code: [Select]
  if (newData == true) {
   
    if (receivedChars[24] != '.') // verify if the last section contains a valid terminator . before the endmarker
    {
      Serial.println("Missing .");
      receivedChars[24] = '.';
    }
   
    strcpy(tempChars, receivedChars);

is not checking the array that is being parsed.

Also you can't be certain that the end of the array is at index 24. You need to check the length to get an index to the end - remembering that the last element will be [length-1]

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Deva_Rishi

Honestly i think by now i would want to get rid of the strtok() function within the parsing all-together, and create my own version of it, that copies and adds to a char* (or even a String, hey you are using an ESP.. ) until it finds a separator or a terminator. This is patch work.
To 'Correct' you have to be Correct. (and not be condescending..)

brice3010

This code
Code: [Select]
 if (newData == true) {
    
    if (receivedChars[24] != '.') // verify if the last section contains a valid terminator . before the endmarker
    {
      Serial.println("Missing .");
      receivedChars[24] = '.';
    }
    
    strcpy(tempChars, receivedChars);

is not checking the array that is being parsed.

Also you can't be certain that the end of the array is at index 24. You need to check the length to get an index to the end - remembering that the last element will be [length-1]

...R
Right, the check should be on tempchars, not receivedchars:
Code: [Select]

if (tempchars[24] != '.') // verify if the last section contains a valid terminator . before the endmarker
    {
      Serial.println("Missing .");
      tempchars[24] = '.';
    }


Indeed, looking at the serial output it seems that the . is not where it was supposed to be, especially because placing a . at that position causes a crash.
So before performing above verification there also needs to be a length verification (although I would not know why the length is not constant given the specific nature of the data being sent).
Question: how to do this length check? My knowledge of string constructs is next to 0, sorry!


brice3010

Honestly i think by now i would want to get rid of the strtok() function within the parsing all-together, and create my own version of it, that copies and adds to a char* (or even a String, hey you are using an ESP.. ) until it finds a separator or a terminator. This is patch work.
I appreciate your input, but my knowledge of writing parsing code is next to 0. And since the work by Robin2 is a major contribution to this forum I would not want to bypass him for this subject.

Robin2

So before performing above verification there also needs to be a length verification (although I would not know why the length is not constant given the specific nature of the data being sent).
Question: how to do this length check? My knowledge of string constructs is next to 0, sorry!
You have to keep in mind that the Arduino has no control over the data it receives. There could be interference with the signal that causes bytes to be misinterpreted (not likely, but possible). And at any one time the received data might include bytes that arrived after the terminator for the previous message. For example if the terminator was '>' and a carriage-return and line-feed followed it.

You can use the strlen() function to determine the length - but read the documentation carefully so you know exactly what it is telling you. Then, if you feel it is necessary, you could iterate over the entire message byte by byte to make sure there are no unexpected values. Have you tried printing the received message just before parsing it?

I can think of no good reason to abandon the strtok() function. It has been tried and tested and found satisfactory millions of times.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

brice3010

#40
Jan 18, 2019, 02:17 pm Last Edit: Jan 18, 2019, 02:27 pm by brice3010
You have to keep in mind that the Arduino has no control over the data it receives. There could be interference with the signal that causes bytes to be misinterpreted (not likely, but possible). And at any one time the received data might include bytes that arrived after the terminator for the previous message. For example if the terminator was '>' and a carriage-return and line-feed followed it.

You can use the strlen() function to determine the length - but read the documentation carefully so you know exactly what it is telling you. Then, if you feel it is necessary, you could iterate over the entire message byte by byte to make sure there are no unexpected values. Have you tried printing the received message just before parsing it?
Good idea. Will be done this evening, results in a day.

This code I will insert right before the parsing starts:

Code: [Select]

  for(int i = 0; i < 32; i++)
  {
    Serial.print(tempChars[i]);
  }
  Serial.println();




I can think of no good reason to abandon the strtok() function. It has been tried and tested and found satisfactory millions of times.

...R
Your contribution on the subject I consider the best I found anywhere.

Robin2

This code I will insert right before the parsing starts:

Code: [Select]

  for(int i = 0; i < 32; i++)
  {
    Serial.print(tempChars[i]);
  }
  Serial.println();



If you add the line
Serial.print(' ');
to put a space between characters it may make the data easier to read - unless there are space characters in the data

Another idea is to change
Code: [Select]
Serial.print( tempChars[1] );
to
Code: [Select]
Serial.print( (byte) tempChars[i] );
which will show the Ascii values in case there are some non-printing characters

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

Deva_Rishi

I can think of no good reason to abandon the strtok() function. It has been tried and tested and found satisfactory millions of times.
But now it is causing a problem since it reads beyond the terminating NULL. (as far as we can tell) So i would want to add a 'do not read beyond size of string' to it, as far as we know it is the cause of the crashes (on the ESP)
Quote
since the work by Robin2 is a major contribution to this forum
I totally agree with that. I also find the Serial transmit-receive examples very good ! (though incomplete imo.)
To 'Correct' you have to be Correct. (and not be condescending..)

brice3010

If you add the line
Serial.print(' ');
to put a space between characters it may make the data easier to read - unless there are space characters in the data

Another idea is to change
Code: [Select]
Serial.print( tempChars[1] );
to
Code: [Select]
Serial.print( (byte) tempChars[i] );
which will show the Ascii values in case there are some non-printing characters

...R
Great!! I will serial print both.

brice3010

But now it is causing a problem since it reads beyond the terminating NULL. (as far as we can tell) So i would want to add a 'do not read beyond size of string' to it, as far as we know it is the cause of the crashes (on the ESP)
Ok, that is what we are trying to figure out now for sure, see my previous post; results in a day or so.

Go Up