Send larger array via Serial buffer – option for less memory usage needed

Projects Programming
1

Hello,

I need to send a 496 byte sized 2d array via the serial port.
I know the workout via enlarging the serial buffer and Im aware of the downside that it takes a lot of ram.
So how can this be avoided? Sending the array content as for example 8 packages á 64 byte ?
If yes, something to take care of when doing this? Checksum or similair?

Thanks for your assistance!
A.

2

Hello

You can send chunks of data and use flush() after each chunk

What is the array content ? Maybe there are ways to reduce its size

Edit: you can also send all your array at once because when the tx buffer is full, write() will wait until there is room in it ( ArduinoCore-avr/HardwareSerial.cpp at master · arduino/ArduinoCore-avr · GitHub ) . I'm not sure which way is faster

3

On the serial line everything will be… serialized
So just send the bytes using some protocol that makes it possible for the receiver to détect the start and end and coherence of the transmission

4

flush() will return when the serial send buffer is empty, write() will return as soon as there is sufficient space in the serial send buffer for the last byte of data being written, so presumably write() will return faster.

Really not a valid question, because flush() does not itself send any data, it just acts as a delay until send buffer empty.

5

Send FROM the Arduino, or send TO the Arduino?
The former doesn't need any special handling; just output the data in chunks as it is computed. For sending TO the arduino, things can get more complicated. It's difficult to handle a 500byte "message" in less than 500 bytes (but you shouldn't need to increase the Serial buffers; just read it as it shows up...)

6

Hi Shannon,

both ways.

BR

7

Hi gulx,

the array is byte ordered, so no reduzing of its size possible. Its an array for an engine spark mapping.

I did an approach by myself, it sends the whole source data amount in 64byte chunks ... any hints for optimisation welcome.
Also a checksum can be send as final byte.

void SerialWriteEx( HardwareSerial *mySerial, byte* pData, int lengthInBytes, bool checksum = false) {
  int i;
  byte cs = 0;

  // Comply with default TX buffer size of 64 bytes

  for ( i = 0; i < (lengthInBytes - (lengthInBytes % 64)); i += 64) {
    mySerial->write(  pData + i, 64);
    mySerial->flush();
  }
  if ( lengthInBytes % 64 ) {
    mySerial->write(  pData + i, lengthInBytes % 64);
    mySerial->flush();
  }

  if (checksum == true) {
    for ( i = 0; i < lengthInBytes; i++) {
      cs ^= *(pData + i);
    }
    mySerial->write(cs);
  }
}

Works fine.

8

As explained sending in chunks does not do anything magic, it’s just bytes sent one after the other, so you can do

mySerial->write( pData, lengthInBytes);

If you have more data than the size of the outgoing buffer, the write() will block until enough data is sent,
There is no need for the flush before the checksum

9

Hi J-M-L,

I see .... so my approach makes no sense. Thanks for again clearifying.

In the very past I had once an issue with a structure in memory larger than the buffer to be send via Serial, and the transfer was cut at the end.
So as I saw many posts in the www how people avoid this by simply enlarging the buffer size in the serials code part header and as this takes on the other hand more memory, .... I thought the "transfer in chunks" approach would handle it.

But finally ... I Just testet it as you described in your last post and all works fine, thnaks!
(Still wonderng why then many people do modify the SoftwareSerial.h to enlarge the buffer if not really needed)

10

Sending the data is usually not the problem but getting it on the other side

most people struggling with receiving the data are usually writing poor code, making second guesses about the timing or not reading fast enough and then the incoming buffer overflows.

Also Software serial is error prone when you try to do other things that uses interrupts. Going to hardware serial is often part of the solution

11

ThanksJ-M-L
and yes, I use HardwareSerial.