Arduino ZERO (SAMD21) SERIAL1 Buffer size

Hello,

a short describtion of the project and problem:

I have an SAMD21 chip (Arduino Zero) which is connected over the normal "Serial" to the Arduino ISE on the PC by USB, and on the SERIAL1 there is a SIM868 connected (same as SIM800 just with GPS) over nomral UART RX/TX Baudrate 115200.

The problem is the RX Buffer on the Zero of the Serial1 is to small, looks like it is only 64 bytes.

In my project the Zero is in sleep_mode and before it can wake up the RX Buffer is already full and I am loosing data coming from the SIM868.

SO I would like to change the SERIAL1 RX Buffer size..... I already found a Serial_Buffer_size which is 64 in the Ringbuffer.ccp/.h .... already changed this in my code to

#define SERIAL_BUFFER_SIZE 256

and also call it back for a checkup

Serial.println(SERIAL_BUFFER_SIZE); ==> so here it shows 256

But nothing changed for SERIAL1 ,...the RX buffer SIZE of SERIAL1 is still 64 bytes..... so there must be another variable to change ....

Please help ....

Some background , Serial1 is already defined default in the variants.cp /.h for the zero .... didn´t make any changes here.

So it would be perfect if you can tell me the file.... or variable i have to change so the Serial1 RX Buffer becomes bigger than 64 bytes.

Best regards,

Andi

changed this in my code to
#define SERIAL_BUFFER_SIZE 256

WHERE did you put that #define? Adding it to your .ino file won't have any effect on the uart library; you have to change it IN RingBuffer.h for it to have any effect. (not designed very well, IMO.)

Hey , thanks ....

So first I just put the #define SERIAL_BUFFER_SIZE 256

in the .ino file before the setup.

And also during the setup I used Serial.println(SERIAL_BUFFER_SIZE); ==> so here it shows 256

But I also tried to change the SERIAL_BUFFER_SIZE directly in the Ringbuffer.h / cpp .... but nothing changed for SERIAL1

That´s why I think there will be another "SERIAL_BUFFER_SIZE" variable for Serial1 ... and the SERIAL_BUFFER_SIZE variable in the Ringbuffer.h/cpp is maybe only for the normal "USB Serial".

So any idea where there might be another SERIAL_BUFFER_SIZE definition for SERIAL1 ... best regards

Hello,

Hate to reopen this but it seems as if it was never really solved. @bembu_account, did you manage to figure out how to change the Serial1 TX_BUFFER_SIZE ?

First off I am using a Mac.

The regular advice has been to change the Serial_TX_BUFFER_SIZE in HardwareSerial.h within "/Applications/Arduino.app/Contents/Java/hardware/arduino/avr/cores/arduino", however, this doesn't decrease the amount of SRAM or change anything when Serial.availableForWrite() is called. Therefore, it doesn't seem like Arduino is even using this path. Obviously though since I am using an SAMD21 board I found some advice to find the path Arduino is reading from by choosing a sketch example under my board, okay good.

As I said I also am using a SAMD21 board (feather m0) and the HardwareSerial.h file under "/Users/userName/Library/Arduino15/packages/adafruit/hardware/samd/1.2.1/cores/arduino"
seems to use the native HardwareSerial.h file:

#ifndef HardwareSerial_h
#define HardwareSerial_h

(as I cannot find any other place to change the Serial TX_BUFFER_SIZE). I do see an option to change the SERIAL_BUFFER_SIZE in RingBuffer.h (164 currently). And by changing that at least the SRAM went down so I know I am in the right directory but that doesn't seem to offer an option to modify the TX buffer.

I've also seen that by the version I am on (1.8.5) apparently you can also change the TX buffer sizes for Serial/Serial1 independently from one another, is this true? That also is important for me.

Thanks

/Applications/Arduino.app/Contents/Java/hardware/arduino/avr/cores/arduino"

Of course changing the AVR UART driver doesn't change the SAMD buffering...

I also am using a SAMD21 board (feather m0) and the HardwareSerial.h file under "/Users/userName/Library/Arduino15/packages/adafruit/hardware/samd/1.2.1/cores/arduino"

That's more like it. However, it seems that the Adafruit SAMD code doesn't HAVE a tx buffer at all.
This is something that was added to the Arduino core about a year ago, and it looks like Adafruit never picked up the edit... (TX buffering for UART using RingBuffer · arduino/ArduinoCore-samd@b24c9e5 · GitHub)

I've submitted UART has no transmit buffer. · Issue #59 · adafruit/ArduinoCore-samd · GitHub
And a similar bug for Sparkfun. Sigh. This is the downside of making it easier to provide 'new' cores rather than derivative cores.

Note that Serial.write() on many SAMD boards goes direct to USB without using the uart driver. (not true of Serial1, of course...)

Hi

The main issue with serial on microcontrollers and the Arduino software is there is no flow control implemented, so everything works okay whilst bytes transferred are bursty in nature and less than the buffer size, but for continuous streams of data or larger amounts of data buffer overflows and buffer underruns are inevitable.

Buffer underruns, where we expect to have some data but find it isn't there yet and so the buffer contains nothing we tend to deal with as a matter of course with Serial.available(). Transmitting, then we generally don't check in the same way and just tend to get away with it, unless we start exceeding the TX buffer that is, or exceed the RX buffer on the destination device.

One option is before sending data is check how much can be sent into the buffer using Serial.availableForWrite() and only write the number of bytes available in the buffer, then do something else until more can be written.

The receiving device may support XON/XOFF, so will send an XOFF byte back if its own buffers are full and the data can't be processed, so the microcontroller could look for that and stop sending if that byte is received, and then start sending again when XON is received.

Personally, I think it is better to deal with the problem by implementing some sort of check, such as availableForWrite, rather than hide the problem in larger buffers, which may not help too much as it could just push the problem to the receiving device where it starts to receive too much data and so its receive buffer overflows and data is discarded.

Regards

Phil

Transmitting, then we generally don't check in the same way and just tend to get away with it, unless we start exceeding the TX buffer that

Huh? Serial.write() (and everything that calls it) is a blocking call. If you send more data than fits in the output buffer, the call just waits until there IS room in the buffer. (Serial.read() is non-blocking, which is inconsistent, and causes many users some confusion, but if you're going to be inconsistent, this is the correct way to do things.) (Note that the size of the serial output buffer in Sparkfun on Adafruit SAMD boards is effectively ONE byte!)

Thanks for trying Phil. And definitely thanks to westfw for adding the issue and recognizing that Adafruit hasn't picked up on the change.

Personally, I think it is better to deal with the problem by implementing some sort of check, such as availableForWrite, rather than hide the problem in larger buffers,

As westfw was saying The Serial1.availableForWrite() on my Adafruit feather m0 always returns 1 byte, this is pretty ridiculous considering I bought it for the 32KB RAM.

Thanks again!

Sorry but the problem is not the TX Buffer of the SAMD... so sending data is not the problem.....

The problem is receiving data from another chip ON SERIAL 1.... "ready/status calls" are not possible so the best solution so far for me would be a change of the RX-Buffer size of SERIAL 1.

westfw you mentioned that " the Adafruit SAMD code doesn't HAVE a tx buffer at all."

.... but i guess there needs to be a RX Buffer size, right???

The normal "SERIAL" is not the problem , the only problem is receiving data on SERIAL1 , the buffer is 64bytes .... so this size has a declaration somewhere....

Maybe anybody has another idea where I can find it???

Not sure about that @bembu_account.

But @westfw, if I try to make the changes (Add the commits that Arduino made to my Adafruit files) I end up running into issues with other methods not existing/being defined. Is there anyway I can fix this rather than being a sitting duck and waiting for Adafruit's developers to get around to it?

Thanks again.

Hi stringEM,

The addition of a serial Tx buffer to the Arduino SAMD core was proposed and implemented by Sandeep Mistry (samdeepmistry) way back in October 2015, but the change wasn't built into the core until July 2017. Which might explain why Adafruit haven't incorporated this change into their SAMD core code just yet.

If you're transmitting a lot of serial data then it's certainly worth adding, as it prevents the code from blocking each time you need to transmit a new character and really improves performance.

I was in the same position back in 2016 and ended up up just cutting 'n' pasting Sandeep's changes into my own copy of the Arduino SAMD core code.

The Github pull request changes required to add the Tx buffer are detailed here: Add TX buffer for Serial by sandeepmistry · Pull Request #58 · arduino/ArduinoCore-samd · GitHub.

Other additional serial port functionality that was also added at the same time (July 2017), and that may or may not be implemented by Adafruit, (I haven't checked), are detailed here: Add TX buffer for Serial by sandeepmistry · Pull Request #58 · arduino/ArduinoCore-samd · GitHub.

Hi bembu_account,

The serial buffer size is defined in the Arduino SAMD core file "RingBuffer.h":

#define SERIAL_BUFFER_SIZE 64

Adafruit have increased the Rx buffer on their boards to 164 bytes.

@westfw,

This is the downside of making it easier to provide 'new' cores rather than derivative cores.

I couldn't agree more. It certainly would be better Arduino, Sparkfun, Adafruit, etc..., provided one common core rather than providing their own similar, but gradually diverging ones.

I guess it's simply easier and more convenient to make changes to their own forked code, rather than submit changes to somebody elses.