Arduino Forum

Using Arduino => Programming Questions => Topic started by: Gennady on Mar 21, 2019, 07:04 am

Title: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 21, 2019, 07:04 am
Hi!

IDE 1.8.2, Nano, parsing lib https://github.com/bolderflight/UBLOX/tree/master/src

When parsing data coming from a GPS NEO M8N 200mS update, there are constant failures or data in general do not pass parsing at a packet size of more than 240 bytes (at speeds of 19200, 38400, 57600).I need 290 bytes! At other speeds I did not check, since they do not suit me. I tested with the revised HardwareSerial.h file with the RX buffer increased to 330 bytes.
Compiling a sketch gives: Global variables use 802 bytes (39%) of dynamic memory, leaving 1246 bytes for local variables. Maximum is 2048 bytes.

Code: [Select]

#if !defined(SERIAL_TX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64 //64
#endif
#endif
#if !defined(SERIAL_RX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_RX_BUFFER_SIZE 330 //16
#else
#define SERIAL_RX_BUFFER_SIZE 330 //64
#endif
#endif



The program in the cycle consists of two parts. The first part is the parsing of the 100 byte portion of the packet and the writing of values ​​into variables. After that, a trigger is set up that allows the operation of another part - my code.
My code does some transformations with the variables obtained above and produces the result on the UART TX. After that, the trigger is reset. Thus, my code is executed only after receiving new data through  library.

Code: [Select]


 
loop{

if (gps.readSensor())
    { digitalWrite(6, HIGH);
      Longitude_deg = gps.getLongitude_deg();
      Latitude_deg  = gps.getLatitude_deg();   
      GnssFixOk     = gps.isGnssFixOk();
      startLatitude_rad  = gps.getLatitude_rad();
      startLongitude_rad = gps.getLongitude_rad();
      MSLHeight_m   = gps.getMSLHeight_m();
      NumSatellites = gps.getNumSatellites();
      triggerParse = 1;
      digitalWrite(6, LOW);
    }

if (  triggerParse == 1 )
    { digitalWrite(9, HIGH);

//my code here....

      triggerParse = 0;
      digitalWrite(9, LOW);

    }
}



 Please advise what I can try to do?


I also give a picture for the case when everything works with a packet length of 240 bytes- channels from top to bottom: parsing work, UART RX, my code work, UART TX.


Thanks in advance!

Gennady
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 21, 2019, 09:24 am
Please post a complete program.

What is the purpose (or intended purpose) of the stuff in the first code block in your Original Post?

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Deva_Rishi on Mar 21, 2019, 10:46 am
what type of variable do you use for parsing ? In fact yes post the complete sketch or at least the parsing functions...
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 21, 2019, 11:43 am


 I tested with the revised  file from //hardware/avr/arduino/ HardwareSerial.h with the RX buffer increased to 330 bytes:

Code: [Select]

#if !defined(SERIAL_TX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64 //64
#endif
#endif
#if !defined(SERIAL_RX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_RX_BUFFER_SIZE 330 //16
#else
#define SERIAL_RX_BUFFER_SIZE 330 //64
#endif
#endif



sketch
Code: [Select]

#include  "UBLOX.h"
#define DEBUG
#define UART_SPEED       57600

double   startLatitude_rad       = 0;
double   startLongitude_rad      = 0;
bool     GnssFixOk          = false;
double   Latitude_deg       = 0;
double   Longitude_deg      = 0;
double   Latitude_rad       = 0;
double   Longitude_rad      = 0;
double   MSLHeight_m        = 0;
uint8_t  NumSatellites      = 0;
double   GroundSpeed_ms     = 0;
double   pointLatitude_rad  = 0;
double   pointLongitude_rad = 0;

byte   triggerParse = 0;

UBLOX gps(Serial, UART_SPEED);


void setup()
{

  pinMode(6, OUTPUT);
  pinMode(7, OUTPUT);
  pinMode(8, OUTPUT);
  pinMode(9, OUTPUT);

  gps.begin();

#ifdef DEBUG
  Serial.println("START ");
  Serial.println("-------------------------- ");
#endif



}

void loop(){

  if (gps.readSensor())
  { digitalWrite(6, HIGH);
    Longitude_deg = gps.getLongitude_deg();
    Latitude_deg  = gps.getLatitude_deg();
    GnssFixOk     = gps.isGnssFixOk();
    startLatitude_rad  = gps.getLatitude_rad();
    startLongitude_rad = gps.getLongitude_rad();
    MSLHeight_m   = gps.getMSLHeight_m();
    NumSatellites = gps.getNumSatellites();
    triggerParse = 1;
    digitalWrite(6, LOW);
  }

  if (  triggerParse == 1 )
  { digitalWrite(9, HIGH);

#ifdef DEBUG
    Serial.print("N ");    Serial.print(NumSatellites);
    Serial.print(" F ");  Serial.print(GnssFixOk);
    Serial.print(" Lat ");  Serial.print(Latitude_deg, 7);
    Serial.print(" Lon ");  Serial.print(Longitude_deg, 7);
    Serial.print(" Spd "); Serial.print(GroundSpeed_ms);
#endif

    triggerParse = 0;
    digitalWrite(9, LOW);

  }
}


 Please advise what I can try to do?
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 21, 2019, 11:49 am
what type of variable do you use for parsing ? In fact yes post the complete sketch or at least the parsing functions...
in UART sending only bytes. Parsing also byte per byte- pasing function from .cpp file from lib:
Code: [Select]


/* reads packets from the uBlox receiver */
bool UBLOX::readSensor()
{
if (_parse(_ubxNavPvt_msgClass,_ubxNavPvt_msgId,_ubxNavPvt_msgLen)) {
return true;
} else {
return false;
}
}

/* parse the uBlox data */
bool UBLOX::_parse(uint8_t msg_class,uint8_t msg_id,uint16_t msg_length)
{
// read a byte from the serial port
while (_bus->available()) {
_byte = _bus->read();
// identify the packet header
if (_parserState < 2) {
if (_byte == _ubxHeader[_parserState]) {
_parserState++;
} else {
_parserState = 0;
}
} else {
if ((_parserState - 2) < msg_length) {
*((uint8_t *) &_tempPacket + _parserState - 2) = _byte;
}
_parserState++;
// compute checksum
if ((_parserState - 2) == msg_length) {
_calcChecksum(_checksum,((uint8_t *) &_tempPacket),msg_length);
} else if ((_parserState - 2) == (msg_length + 1)) {
if (_byte != _checksum[0]) {
_parserState = 0;
}
} else if ((_parserState - 2) == (msg_length + 2)) {
_parserState = 0;
if (_byte == _checksum[1]) {
memcpy(&_validPacket,&_tempPacket,sizeof(_validPacket));
return true;
}
} else if (_parserState > (msg_length + 4) ) {
_parserState = 0;
}
}
}
return false;
}

/* uBlox checksum */
void UBLOX::_calcChecksum(uint8_t* CK, uint8_t* payload, uint16_t length)
{
CK[0] = 0;
  CK[1] = 0;
for (uint8_t i = 0; i < length; i++) {
CK[0] += payload[i];
CK[1] += CK[0];
}
}

Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 22, 2019, 03:42 pm

Really nobody can advise?
Maybe this is a problem with the Stream class.
Or do I increase the buffer in 1.8.2, and the IDE takes the Hardwareserial.h file from 1.6.5, which is also installed by me?
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 22, 2019, 04:00 pm
Or do I increase the buffer in 1.8.2, and the IDE takes the Hardwareserial.h file from 1.6.5, which is also installed by me?
As a piece of English that makes no sense.

And if you have two versions of the IDE on your PC they will be completely separate. You can use whichever you choose. Recently when I needed a bigger serial input buffer I just made a copy of the IDE with a different name and changed the buffer size in that, Now I can use whichever version suits my requirement.

If you have a buffer that is bigger than the message that is being sent then I can think of no technical reason why the message cannot be parsed. However I know nothing about the UBLOX library you are using and that is where I would start searching for the problem.

Please post a link to the datasheet with the specification for the data that is SENT by the GPS device.

An example of a typical message would also be a big help. If it contains non-printing characters be sure to identify them all. They can be the key to effective parsing.

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 22, 2019, 05:24 pm
Thanks for the answer!
Oddly enough, the Serial library at compilation was taken from 163. I deleted 163. This did not help.

Today I experimented with the messages in the package. NEO 57600baud, 5Hz. The working version is the following and the only one.

UBX NAV-SOL, Size 60, 'Navigation Solution'
UBX NAV-PVT, Size 100, 'Navigation PVT Solution'
UBX NAV-POSLLH, Size 36, 'Geodetic Position'
UBX NAV-VELNED, Size 44, 'Velocity in WGS 84'

Total 240 bytes.

In this order, NEO builds them.

I add MON-HW2 long 36 byte. He stands in front of SOL on some time lead. But its frequency of appearance is 1 Hz. At this point, the parsing does not work. After that, I removed MON-HW2, but I inserted the message DOP 26 byte. It fell AFTER the POSLLH message. With him, the parsing also disappeared. At lower speeds 38400, 19200 the same.


I noticed that after pressing the reset button, the first parsing takes place under any conditions.
It seems that with a packet of more than 240 bytes, some kind of buffer overflows.

Maybe problem in Stream.h lib?

I do not know what to do anymore ((((



https://www.u-blox.com/sites/default/files/products/documents/u-blox8-M8_ReceiverDescrProtSpec_%28UBX-13003221%29_Public.pdf
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 22, 2019, 06:33 pm
From a quick look at Section 32 of the linked PDF it seems that a message has the style
$GPGLL,4717.11634,N,00833.91297,E,124923.00,A,A*6E
and is terminated with a CR/LF

I would be surprised if any message is longer than 64 bytes - but you might like to point out which messages may be longer.

I reckon the second example in Serial Input Basics (http://forum.arduino.cc/index.php?topic=396450.0) should receive those messages and the parse example can be adapted to do the parsing.

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 22, 2019, 06:51 pm
From a quick look at Section 32 of the linked PDF it seems that a message has the style
$GPGLL,4717.11634,N,00833.91297,E,124923.00,A,A*6E
and is terminated with a CR/LF

I would be surprised if any message is longer than 64 bytes - but you might like to point out which messages may be longer.

I reckon the second example in Serial Input Basics (http://forum.arduino.cc/index.php?topic=396450.0) should receive those messages and the parse example can be adapted to do the parsing.

...R
No! It NMEA protocol. I use only UBX! Message some like this (up to 100byte and next Message start w/o any CR/LF :

//UBX-NAV-VELNED   
     
16:11:32  0000  B5 62 01 12 24 00 00 8B 39 1D 07 00 00 00 EC FF  µb $ 9 ìÿ
              0010  FF FF DF FF FF FF 27 00 00 00 15 00 00 00 71 C2    ÿÿßÿÿÿ' qÂ
              0020  D1 01 83 00 00 00 3C 01 47 00 2C 60                    Ñ < G ,`

//UBX-NAV-DOP 
16:11:33  0000  B5 62 01 02 1C 00 90 8C 39 1D 53 96 4D 12 96 DF  µb 9 S M ß
              0010  05 1E 9B FC 01 00 0A 99 01 00 36 15 00 00 13 1D    ü 6
              0020  00 00 28 3D                                                        (=

total packet up to 300byte and more.
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Deva_Rishi on Mar 22, 2019, 08:05 pm
And if you have two versions of the IDE on your PC they will be completely separate. You can use whichever you choose. Recently when I needed a bigger serial input buffer I just made a copy of the IDE with a different name and changed the buffer size in that, Now I can use whichever version suits my requirement.
I thought i saw a way passing by on this forum that just puts an altered version of HardwareSerial.h in the project folder and the compiler will use that over the normal one.
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 22, 2019, 11:43 pm
I thought i saw a way passing by on this forum that just puts an altered version of HardwareSerial.h in the project folder and the compiler will use that over the normal one.
I think I spent an hour with Google trying to figure that out and then the simple solution occurred to me - took all of 5 minutes.

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 22, 2019, 11:44 pm
No! It NMEA protocol. I use only UBX!
What section of the PDF is that in ?

Why do you seem to be using the more difficult system?

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 23, 2019, 04:27 am
Thank you for your attention to my problem! Below are links to pages with descriptions of individual messages. The format UBX is good, compact, belongs to Ublox, I use it because it is used in autopilot.
NMEA it is very old format for ships. All the time I think that the problem is not in the program, but in the buffer or something else

33.17 UBX-NAV (0x01) ... 301

33.17.5 UBX-NAV-DOP (0x01 0x04) ........................................................................................ 304

33.17.13 UBX-NAV-POSLLH (0x01 0x02).................................................................................... 312
33.17.14 UBX-NAV-PVT (0x01 0x07) ........................................................................................ 313
33.17.20 UBX-NAV-SOL (0x01 0x06) ........................................................................................ 322
33.17.31 UBX-NAV-VELNED (0x01 0x12) .................................................................................. 337


I gave the parsing code above. As for me, everything is right there.
I read somewhere that Serial uses the Stream lib and the problem with receiving big data in the restrictions in Stream. "Some of the libraries that rely on Stream include : Serial..." Or the problem may be that the Serial lib buffer in the CPU RAM is full as a result of the program execution. And new data does not come. This can be confirmed by the fact that after pressing the reset button ALWAYS ONE parsing occurs. And then they are not, or there is very rarely, once a minute. This is with a packet of more than 240 bytes !!! Can a program in a loop have to clear buffer in the CPU RAM by a command Serialend () or another command? I do not know how well the Serial library is written for this case.
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Robin2 on Mar 23, 2019, 07:48 am
The UBX frame structure is described on pages 134/5

With that type of message you need code to detect the start characters and then to interpret the length of the payload.


I don't know that I can give you any more assistance.

...R
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: MorganS on Mar 23, 2019, 01:33 pm
I would start by modifying the library to print every character that it gets. Plus print the state. That would help answer the following questions...

1. After the GPS has emitted its first valid packet, does it keep sending more valid packets?

It may need a command or acknowledgement to send the next one.

2. Does the parser get stuck in a stable state? Stable is bad here. Any sequence of invalid characters should eventually get it back into the zero state that is looking for the start character.

3. Is there something between the packets like carriage-return putting it into that stable state?

4. Is it overruning any buffer and writing received characters to places it shouldn't?

I've never worked with that library but it does seem unlikely that it is so terrible that the parser gets stuck like that.
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 23, 2019, 01:35 pm
With that type of message you need code to detect the start characters and then to interpret the length of the payload.

...R
p.313 UBX-NAV-PVT
                                  Header      Class ID      Length (Bytes)    Payload      Checksum
Message Structure     0xB5 0x62 0x01 0x07    92                     see below   CK_A CK_B

From lib UBLOX.h:
Code: [Select]

const uint8_t _ubxHeader[2] = {0xB5, 0x62};
    const uint8_t _ubxNavPvt_msgClass = 0x01;
    const uint8_t _ubxNavPvt_msgId = 0x07;
    const uint16_t _ubxNavPvt_msgLen = 96;//total data 92+header 2+ mess class/type 2=96

 

Parsing From lib UBLOX.cpp::
Code: [Select]



/* parse the uBlox data */
bool UBLOX::_parse(uint8_t msg_class,uint8_t msg_id,uint16_t msg_length)
{
 // read a byte from the serial port
 while (_bus->available()) {
 _byte = _bus->read();
 // identify the packet header
 if (_parserState < 2) {
 if (_byte == _ubxHeader[_parserState]) {
 _parserState++;
 } else {
 _parserState = 0;
 }
 } else {
 if ((_parserState - 2) < msg_length) {
 *((uint8_t *) &_tempPacket + _parserState - 2) = _byte;
 }
 _parserState++;
 // compute checksum
 if ((_parserState - 2) == msg_length) {
 _calcChecksum(_checksum,((uint8_t *) &_tempPacket),msg_length);
 } else if ((_parserState - 2) == (msg_length + 1)) {
 if (_byte != _checksum[0]) {
 _parserState = 0;
 }
 } else if ((_parserState - 2) == (msg_length + 2)) {
 _parserState = 0;
 if (_byte == _checksum[1]) {
 memcpy(&_validPacket,&_tempPacket,sizeof(_validPacket));
 return true;
 }
 } else if (_parserState > (msg_length + 4) ) {
 _parserState = 0;
 }
 }
 }
 return false;
}

/* uBlox checksum */
void UBLOX::_calcChecksum(uint8_t* CK, uint8_t* payload, uint16_t length)
{
 CK[0] = 0;
  CK[1] = 0;
 for (uint8_t i = 0; i < length; i++) {
 CK[0] += payload[i];
 CK[1] += CK[0];
 }
}





From you, I want to get advice on the work of the Stream and Serial libraries from the Arduino core. The problem is in them !!!!! This is indicated by the fact that after a reset, the first time parsing is ALWAYS correct once. Then not or rarely! The buffer of the Serial library
(filled independently of the program cycle through ISR UART) is overflowed with data and new data AT ALL do not arrive. How to clear this buffer? Or a problem in Stream with a large amount of data. Here I need your great experience!
Thanks in a
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 23, 2019, 02:01 pm
I would start by modifying the library to print every character that it gets. Plus print the state. That would help answer the following questions...

1. After the GPS has emitted its first valid packet, does it keep sending more valid packets?

It may need a command or acknowledgement to send the next one.

2. Does the parser get stuck in a stable state? Stable is bad here. Any sequence of invalid characters should eventually get it back into the zero state that is looking for the start character.

3. Is there something between the packets like carriage-return putting it into that stable state?

4. Is it overruning any buffer and writing received characters to places it shouldn't?

I've never worked with that library but it does seem unlikely that it is so terrible that the parser gets stuck like that.
1. GPS transmits packets continuously. The length of the packet is up to 100 millis, I have 200 millis repetition time (5 Hz). Necessary messages are placed in the package even at a speed of 19200. Leave free time. I need to get only one of the five messages in the package. Separate messages are joined in a packet without intervals. There is no command to transfer, the transfer goes automatically and continuously.
2.

I think the parser works that way. Therefore, with a packet length of up to 240 bytes, everything is fine! More than 240 bytes, it does not get stuck, I think, it does not receive data from the Serial library at all, or it receives very rarely - once a minute.

3. No, first byte of next  messages come.

4.No

Sample of packet with two messages:
12:54:28  R -> UBX NAV-DOP,  Size  26,  'Dilution of Precision'
12:54:28  R -> UBX NAV-VELNED,  Size  44,  'Velocity in WGS 84'


b5 62 01 04 12 00 f8 f1 ad 21 c0 00 a7 00 5e 00  µb....шс­!А.§.^.
85 00 64 00 4b 00 43 00 0a 34 b5 62 01 12 24 00  ….d.K.C..4µb..$.
f8 f1 ad 21 07 00 00 00 05 00 00 00 fa ff ff ff  шс­!........ъяяя
0b 00 00 00 09 00 00 00 7a 26 25 00 64 00 00 00  ........z&%.d...
f2 84 2d 00 d1 ec



Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: MorganS on Mar 23, 2019, 02:05 pm
If there was a problem with Serial or Stream then millions of people would be having that problem.

If the problem is in the ublox library then maybe only a thousand people and they're not posting it here.

Check the characters that the library actually received and acted upon.

Quote
Therefore, with a packet length of up to 240 bytes, everything is fine! More than 240 bytes, it does not get stuck
Which is it?
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 23, 2019, 03:21 pm
If there was a problem with Serial or Stream then millions of people would be having that problem.

If the problem is in the ublox library then maybe only a thousand people and they're not posting it here.

Check the characters that the library actually received and acted upon.
Which is it?
If the packet is larger than 240 bytes, the parsing does not work or it works very rarely, about once a minute. I checked the UBLOX library along with the author. He developed it for T_3.5 / 3.6, everything works for him. Please explain how the Serial library RX buffer works. What happens if it is full, how to clean it?
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: MorganS on Mar 23, 2019, 05:05 pm
If the Serial buffer is full then you have screwed up. The simple way to do that would be to put delay(100) in your program.

I actually do that in the setup() of some of my important projects. There are delays initializing the LCD screen and other peripherals. So I clear the Serial buffer at the end of setup() like this...

Code: [Select]
  while(Serial.available()) Serial.read();
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 23, 2019, 08:27 pm
If the Serial buffer is full then you have screwed up. The simple way to do that would be to put delay(100) in your program.

I actually do that in the setup() of some of my important projects. There are delays initializing the LCD screen and other peripherals. So I clear the Serial buffer at the end of setup() like this...

Code: [Select]
  while(Serial.available()) Serial.read();
Sooner or later, any buffer can be filled with data. It is not always possible to do the parsing often enough. Maybe just the CPU performance is not enough.
If I understand correctly, then the buffer is filled with data by an interrupt from the UART RX. Therefore, it is necessary to make room for new data before starting the parsing in LOOP ().There should be a clear buffer command. After all, this is just zeroing the variable of the buffer filling counter. Can Serial.End () and Serial.began () apply and then enable parsing? Then the parsing will checknew data like in first  packet, as after resetting the CPU.
Of course, I will try also your buffer cleaning solution. Thanks!
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Deva_Rishi on Mar 23, 2019, 10:02 pm
Can Serial.End () and Serial.began () apply and then enable parsing? Then the parsing will checknew data like in first  packet, as after resetting the CPU.
Yes it does, also you can set the pointer to the last byte = pointer to the first byte (Serial.begin() sets both to 0, from the (ring)buffer offset. I think, i read it somewhere but i haven't checked inside hwSerial.cpp, or maybe i did, but haven't just now to verify my memory. You should, you could also add the function to clear the buffer to hwSerial as you think it suits you most.
If the GPS is constantly broadcasting, there should be a break of a certain length in between broadcasts (or at least one would hope so) and you can look for that before you start listening. Since the main part is raw data, it may be hard to look for a header.
Quote
Sooner or later, any buffer can be filled with data. It is not always possible to do the parsing often enough. Maybe just the CPU performance is not enough.
in some way yes, though with you baudrate that should not really be an issue. Either way check out this (https://www.maxpierson.me/2009/03/20/receive-dmx-512-with-an-arduino/) way of receiving DMX on an Arduino, with having the ISR vectors redefined and the ISR's in the Sketch. There is a more modern way using the Conceptinetics DMX library, you can get some more details out of that.
Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: MorganS on Mar 23, 2019, 11:46 pm
Sooner or later, any buffer can be filled with data. It is not always possible to do the parsing often enough. Maybe just the CPU performance is not enough.
No. Serial is really slow. Even the slowest Arduino can perform thousands of operations between each Serial character arriving.

If you are burning up those thousands of instructions doing floating-point calculations then that doesn't mean Serial is broken. It means you or your library are doing something wrong.

Quote
If I understand correctly, then the buffer is filled with data by an interrupt from the UART RX. Therefore, it is necessary to make room for new data before starting the parsing in LOOP ().
Nope. Parsing by using Serial.read() is the only thing that should be taking characters out of that buffer. If parsing is working fast enough then it will empty the buffer before the next packet finishes transmission.

Quote
There should be a clear buffer command. After all, this is just zeroing the variable of the buffer filling counter. Can Serial.End () and Serial.began () apply and then enable parsing? Then the parsing will checknew data like in first  packet, as after resetting the CPU.
Double nope! If you erase the buffer then you would have to wait at least 100 milliseconds for the next packet to start. Then wait for that packet to finish. That is a long time for the Arduino to waste when it already had the data buffered and waiting for the parser.


Quote
Of course, I will try also your buffer cleaning solution. Thanks!
No. Don't use that code. Just verify that the parser always completes its job before the next packet is complete in the Serial buffer.

Print out Serial.available() after getting that first packet. It should be a small number like 0 or maybe 100.

Title: Re: Problem with parsing big packets on Serial port Arduino Nano
Post by: Gennady on Mar 24, 2019, 08:25 am

Thank! I will use your advice and conduct experiments.