Modifying serial buffer size in arduino 1.8.1

Projects Programming
1

In older versions of arduino , in "HardwareSerial.h" , changing #define SERIAL_BUFFER_SIZE 256 is enough . But in arduino 1.8.1 , how to change buffer size to 256 ?

HardwareSerial.h of arduino 1.8.1

#if !defined(SERIAL_TX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_TX_BUFFER_SIZE 16
#else
#define SERIAL_TX_BUFFER_SIZE 64
#endif
#endif
#if !defined(SERIAL_RX_BUFFER_SIZE)
#if ((RAMEND - RAMSTART) < 1023)
#define SERIAL_RX_BUFFER_SIZE 16
#else
#define SERIAL_RX_BUFFER_SIZE 64
#endif
#endif
#if (SERIAL_TX_BUFFER_SIZE>256)
typedef uint16_t tx_buffer_index_t;
#else
typedef uint8_t tx_buffer_index_t;
#endif
#if  (SERIAL_RX_BUFFER_SIZE>256)
typedef uint16_t rx_buffer_index_t;
#else
typedef uint8_t rx_buffer_index_t;
#endif
2

With later versions of the IDE, the TX and RX buffers no longer need to be the same size. Which one do you want to change the size of?

Why?

3

I want to change both buffers , I'n using esp8266 to communicate with server . Hence I need to increase TX buffer to send all command at a time and RX buffer to receive all data at a time .

4

For the TX, I can still imagine the requirement because sending more than the size of the buffer will slow down other parts of the code till there is space in the buffer again.

But for the RX, it's more than likely your code that is the bottleneck.

Show your code so people can give proper advice.

5

If you are concerned about receiving data have a look at the examples in Serial Input Basics - simple reliable ways to receive data. As written it uses a 32 byte array but you can easily change that by changing the constant numChars.

...R

6

In older versions of arduino , in "HardwareSerial.h" , changing #define SERIAL_BUFFER_SIZE 256 is enough . But in arduino 1.8.1 , how to change buffer size to 256 ?

How much "older" are you used to? I don't think the code has been significantly different from the 1.8.x code since 1.0.x (at least, my 1.6.5 tree has about the same code.)

To change it, change the occurrances of "64" to "256"

7

Separate size definitions RX and TX buffers were introduced in 1.5.7.

8

Hi, I also tried changing the buffer size in the .h file, but the IDE
obviouly does not use this file during compile process, i.e. the
resultung binary remains the same. Even deleting the .h file does
not lead to an error.
Of course I disabled the aggressive caching feature and I also
removed all the .o files in the temp binary directory...

Any ideas how to force the IDE to do a complete rebuild?

9

Which .h file did you change?
And what versions do you have installed? (IDE, etc)

10

IDE is 1.8.2 and after reading various threads here and elsewhere I naively thought,
that this file is the right one:

C:\Program Files\Arduino\hardware\arduino\avr\cores\arduino\HardwareSerial.h

But the IDE simply seems to ignore this one (a search using the explorer did not
show up any other files with this name). Even completely removing all the
HardwareSerial.* files does not lead to a compile error (even with all .o files
in TEMP-PATH\Arduino\core removed - so I guess the IDE simply still uses the
precompiled lib and does NOT compile the HardwareSerial files...

11

...yes I thought on changing the code to read more often, but the
project is doing quite some graphics/display work with data coming
in at 110000Baud. So I'd have to query the serial port doing display
work, which is not very esthetic...

Any hints are welcome...

12

NeoHWSerial allows you to attach a function to the RX interrupt. You could use this to parse the received data while the drawing is busy.

Have you tried to break the drawing into several smaller steps, with serial processing in between them?

I usually recommend the latter non-blocking approach when you have control over the code. The former approach may be required when a library is performing some atomic operation (e.g., SD write).

13

I also have got SD write of course... I can not imagine why the "increased buffer" approach is
not working on my side where lot of people had success. Is there a command line to force
compiling the library?

The project is rather complex with lot of IO etc (for a early stage see
FIN-RPMD-Explorer, a tribute to Inertial Navigation in early Tornado - YouTube) and every change in the tool chain or
introduction of new interrupts/levels leads to a lot of work an side-effects
which I want to avoid :wink:

14

Just did a crosscheck: I am normally working under XP in a VM. Repeated the
experiment under native Win7 with 1.8.2 and a sketch using serial and serial3
on Due:

Even removing HardwareSerial.* does not lead to a compile error. Something
scary is happening here I guess...

In which of the libraries of Arduino the HardwareSerial is included (just to have
the chance to check the compile output whether the lib gets built)?

15

Different processors might have different HardwareSerial libraries (and you might have modified/removed the wrong one).

16

I am compiling for the Due and in the output I see a USARTClass.cpp - Hey, that is
the key: This USARTClass.cpp refers to RingBuffer which contains a buffersize of
128 in my installed 1.8.2!

Changing this size leads to a lengthy compile run with a different binary!

Thanks for this decisive hint, Erik.

17

I am compiling for the Due

Arrrrgggghhh. You might have made that clearer, long before post 13 of the thread!

I naively thought, that this file is the right one:
C:\Program Files\Arduino\hardware\arduino\avr\cores\arduino\HardwareSerial.h

Nothing in the "avr" directory is used for Due, which does not have an AVR processor.

In general, use verbose compilation output and search it to see exactly which core code you are using, and then hunt in that directory (and perhaps "nearby")...

18

Yes, that is what I learned during my sessions - well, IMHO it is somewhat strange to
put some source in the Program folder and other soruce in the user's app data - this
is a very strange location! I had expected the arduino source more structured in one directroy
finding avr/ sam/ etc. and everything neede below those?

Most search tools also do not index the app data folder, so no chance to find files
using the OS tools there :frowning:

OK, my fault not to mention Due earlier - thanks for all the help...

19

it is somewhat strange to put some source in the Program folder and other soruce in the user's app data

I agree. But that's apparently what Windows wants, if you're going to have "board manager" type "add boards" capability; can't have user programs in non-admin mode writing to "Program Files", can we?

Most search tools also do not index the app data folder, so no chance to find files using the OS tools there

Probably also considered a "feature" by microsoft. Sigh.
It's a little worse on a Mac, where all the (default) core files and tools are tucked inside the ".app" "file", and the add-ons are off in a "Library" folder, and I think both of those are also "not indexed for search."

Fortunately, if you have the IDE set up to show verbose compilation, it shows the full paths of all the files in the log.

(Also, you can do "portable" installs, in which case everything is kept under the one directory. But you probably have to put it in a non-standard install location or fiddle with access settings...)