Understanding Serial write sending data bigger than buffer size¸

Projects Networking, Protocols, and Devices
1

Hello,

I would like to ask a question regarding the atmega328p's serial output buffer. I want to send for example 150 bytes of data over the serial port and read the data on the other end (PC), where I am reading the data sent every 5 ms.

The relevant code on the uC:

int16_t some_data[150];

void setup() {
	Serial.begin(115200);
}

void loop() {

	uint8_t* pD = (uint8_t*) &some_data[0];

	for(int16_t i = 0;i<150;i++,pD++) {
		Serial.write(*pD);
	}
}

Is it possible to write the data to the serial port like this? Since there is no delay, will this overflow the buffer? Because, sometimes I get the data ok, but sometimes the data seems to be corrupt.

Do you have any advice on this?Can I increase the buffer size of the chip by changing the HardwareSerial.cpp?I am uploading my code through ICSP without the bootloader. Should I sent the data in three packets with a delay between like so:

int16_t some_data[150];

void setup() {
	Serial.begin(115200);
}

void loop() {

	uint8_t* pD = (uint8_t*) &some_data[0];

	for(int16_t i = 0;i<64;i++,pD++) {
		Serial.write(*pD);
	}
	
	pD++;
	delay(some_delay);
	
	for(int16_t i = 0;i<64;i++,pD++) {
		Serial.write(*pD);
	}

	pD++;
	delay(some_delay);

	for(int16_t i = 0;i<22;i++,pD++) {
		Serial.write(*pD);
	}
}

Any other thoughts? Any help would be much appreciated.

Thank you and best regards,
K

2 
int16_t some_data[150];

Thats 300 bytes of data!!

void loop() {

	uint8_t* pD = (uint8_t*) some_data;  // the name of an array is the address of the first element.

	for(int16_t i = 0;i<150;i++) {
		Serial.write(*pD++); // merged the ++ here
	}
}

Since there is no delay, will this overflow the buffer?

Definitely!, you can increase the internal buffer in HardwareSerial.cpp , look for
#if (RAMEND < 1000)
#define SERIAL_BUFFER_SIZE 16
#else
#define SERIAL_BUFFER_SIZE 64 <<< change that to 128 or 256 (must be a power of 2)
#endif

or write in parts and use - Serial.flush() - Arduino Reference -

3

Hello,

thank you very much for the quick answer and explanation.

robtillaart:

int16_t some_data[150];

Thats 300 bytes of data!!

I honestly meant an 8bit datatype, sorry for the mistake. :slight_smile:

What are the problems if I don't use the power of 2 for the serial buffer size? My problem is that my SRAM is quite full (~70%) and changing the buffer size to 128 bytes fills it up to ~85%, 256 bytes to 98%. Even 85% seems high to me, but this should work without problems, I think.

Is it possible to increase the Tx buffer size an Rx buffer size individually?

I will try using the 128 bytes buffer and reading faster from the port and see what will happen. Do you know at what rate does the processor fill the data into the output buffer if the is no manual delay? For example if I send 30 bytes of data, and don't read them, how fast will the buffer fill up?

Thank you again and best regards,
K

4

Chimera:
...
What are the problems if I don't use the power of 2 for the serial buffer size? My problem is that my SRAM is quite full (~70%) and changing the buffer size to 128 bytes fills it up to ~85%, 256 bytes to 98%. Even 85% seems high to me, but this should work without problems, I think.

The power of two is a performance issue. The code uses tx_buffer1.tail = (tx_buffer1.tail + 1) % SERIAL_BUFFER_SIZE;
which is optimized when buffersize is a power of 2 to tx_buffer1.tail = (tx_buffer1.tail + 1) & (SERIAL_BUFFER_SIZE -1);
which is much faster. But yes you can use other sizes like 150.

Is it possible to increase the Tx buffer size an Rx buffer size individually?

search for

#if (RAMEND < 1000)
  #define SERIAL_BUFFER_SIZE 16
#else
  #define SERIAL_BUFFER_SIZE 64
#endif

struct ring_buffer
{
  unsigned char buffer[SERIAL_BUFFER_SIZE];
  volatile unsigned int head;
  volatile unsigned int tail;
};

change that too (e.g)

#if (RAMEND < 1000)
  #define TX_SERIAL_BUFFER_SIZE 16
  #define RX_SERIAL_BUFFER_SIZE 16
#else
  #define TX_SERIAL_BUFFER_SIZE 150
  #define RX_SERIAL_BUFFER_SIZE 32
#endif

struct tx_ring_buffer
{
  unsigned char buffer[TX_SERIAL_BUFFER_SIZE];
  volatile unsigned int head;  // could be uint8_t to save a byte per head/tail
  volatile unsigned int tail;
};

struct rx_ring_buffer
{
  unsigned char buffer[RX_SERIAL_BUFFER_SIZE];
  volatile unsigned int head;
  volatile unsigned int tail;
};

If you compile then you will get a list of places where the code is broken due to the changes, fixes should be obvious.

I will try using the 128 bytes buffer and reading faster from the port and see what will happen. Do you know at what rate does the processor fill the data into the output buffer if the is no manual delay? For example if I send 30 bytes of data, and don't read them, how fast will the buffer fill up?

I do not have the timing figures. Just give it a try.

5

Dear Sir,

thank you very much for your exhaustive answer (s)!

I will try your suggestions.

One more question please: if I define the buffer size (not the Tx and Rx individually, but the one defined in the original HardwareSerial.cpp!), does that mean that this size is allocated for both the Tx and Rx buffer?

Thank you again and best regards,
K

6

yes...

7

You can't overflow the outgoing buffer. The Serial.write() function checks that there is room in the buffer. If there is, it stores the byte, and returns immediately, If there is not, the function waits until there IS room.

Since making room requires that interrupts be enabled, this is why you can't expect to be able to call Serial.write() or Serial.print() in an interrupt service routine.

8

Dear Sir,

thank you for the answer.
So if I understand correctly, even if i call serial write 1000 times - when the outgoing buffer gets filled, the serial.write will wait until the buffer is free and only then new data will be written to the buffer? But the rest of the code will execute nevertheless?
What if the next loop iteration occurs before the data gets sent?
If the buffer cannot get overflowed then the only explanation is that the code stops executing until all the bytes are sent?
Please help me understand,sinc you can see i am a bit slow to catch up...

Thank you and best regards,
K