Pages: [1] 2   Go Down
Author Topic: Serial port "rx_buffer" gobbles up unnessary RAM  (Read 2926 times)
0 Members and 1 Guest are viewing this topic.
Valencia, Spain
Offline Offline
Faraday Member
**
Karma: 118
Posts: 4569
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The serial port functions in HardwareSerial.cpp allocate 128 bytes of buffer for receiving data, this seems excessive. 16 bytes would be plenty at serial port data speeds.

At the moment, calling *any* serial port functions will use up 128 bytes of RAM.

Even just "Serial.begin()" will mean the RAM is used up.

Mostly I use the serial port for sending debug output, I never receive any data so this is a BIG overhead.

Logged

No, I don't answer questions sent in private messages...

Left Coast, CA (USA)
Offline Offline
Brattain Member
*****
Karma: 331
Posts: 16540
Measurement changes behavior
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You are free to change it at your liking. the core file to edit is named HardwareSerial.cpp and is located in the arduino cores\arduino directory.

The relevant section is here:

Code:
// Define constants and variables for buffering incoming serial data.  We're
// using a ring buffer (I think), in which rx_buffer_head is the index of the
// location to which to write the next incoming character and rx_buffer_tail
// is the index of the location from which to read.
#if (RAMEND < 1000)
  #define RX_BUFFER_SIZE 32
#else
  #define RX_BUFFER_SIZE 128
#endif

Lefty
Logged

Valencia, Spain
Offline Offline
Faraday Member
**
Karma: 118
Posts: 4569
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Yes, I saw that earlier (that's how I know it's 128 bytes...)

The problem is if I give my code to other people it won't work if they don't edit their copy of HardwareSerial.cpp.

I think it would be a good idea to do something like this in the library:

Code:
#ifndef RX_BUFFER_SIZE

#if (RAMEND < 1000)
  #define RX_BUFFER_SIZE 32
#else
  #define RX_BUFFER_SIZE 128
#endif

#endif
That way people can override the setting without any editing of the library.

Right now I think I have to copy HardwareSerial.cpp into my sketch with a different name and modify it there.

Logged

No, I don't answer questions sent in private messages...

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1481
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You can't overide the buffer size with an #ifndef since the size is set when the library is compiled, not in the context of your sketch.

In Arduino 1.0 there will also be a tx buffer that must be the same size as the rx buffer.

I have my own serial library that allows buffer sizes to be set at run time but, as you say, I can't release code that uses it since its ISR conflicts with programs that use Serial.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 170
Posts: 12480
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


Would it be an idea to propose the buffersize as an optional parameter in

Serial.begin (long baudrate, int buffersize = 128);

// Yes I would like to set stop and startbits too ...

Quote
The problem is if I give my code to other people it won't work if they don't edit their copy of HardwareSerial.cpp.
You could add a readme.txt file in which the mods are described.

question: is your sketch out of RAM?   do you need to recover those 128 bytes?

Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1481
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I like buffer size in begin().  In 1.0 begin() needs to specify RX and TX buffer sizes.

I implemented run-time buffer sizes in my rewrite of HardwareSerial.  I now plan to remove output buffering so I will have begin(baudRate, bufSize).

I agree, it would be nice to set other options like stop bits and parity.

Better error handling would be nice.  Detecting receive overruns would be useful.

I shouldn't complain since I am not actively trying to get these changes into the official distribution.  I just don't have the temperament to do that.  I soon become frustrated with the process.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 170
Posts: 12480
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
Detecting receive overruns would be useful.

That would involve just setting a flag when data is received, not to difficult I think ... (take a deep breath and dive into some code)....

[removed raw thoughts]

See post below for working code
« Last Edit: September 10, 2011, 10:58:49 am by robtillaart » Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Valencia, Spain
Offline Offline
Faraday Member
**
Karma: 118
Posts: 4569
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I like buffer size in begin().  In 1.0 begin() needs to specify RX and TX buffer sizes.

That would need dynamic memory allocation. It seems much more efficient to use #define/#ifdef for this.

Logged

No, I don't answer questions sent in private messages...

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 170
Posts: 12480
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

WRT overflow() I added the lost char counter into the ringbuffer

HardwareSerial.cpp
Code:
struct ring_buffer
{
  unsigned char buffer[RX_BUFFER_SIZE];
  int head;
  int tail;
  long lost;  // <<<< ADDED
};
and
Code:
inline void store_char(unsigned char c, ring_buffer *rx_buffer)
{
  int i = (unsigned int)(rx_buffer->head + 1) % RX_BUFFER_SIZE;

  // if we should be storing the received character into the location
  // just before the tail (meaning that the head would advance to the
  // current location of the tail), we're about to overflow the buffer
  // and so we don't write the character or advance the head.
  if (i != rx_buffer->tail) {
    rx_buffer->buffer[rx_buffer->head] = c;
    rx_buffer->head = i;
  } else rx_buffer->lost++;  // <<<<<<<<<<< ADDED
}

Added this function
Code:
// note: lost counter is not reset ...
uint32_t HardwareSerial::overflow(void)
{
 return  _rx_buffer->lost;
}

Added in hardwareSerial.h the prototype of course.
Code:
virtual uint32_t overflow(void);

Works like a charm!

test sketch, just keep sending chars  smiley
Code:
void setup()
{
  Serial.begin(115200);
}

void loop()
{
  Serial.println(Serial.overflow());
  Serial.println(Serial.available());
  Serial.println("----------");
  delay(1000);
}

reported as issue 637, - http://code.google.com/p/arduino/issues/detail?id=637 -
« Last Edit: September 10, 2011, 11:12:18 am by robtillaart » Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1481
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

It's dead simple to modify HardwareSerial, that's why I have rewritten it the way I like for my own use.

The problem is you can't ask users to replace HardwareSerial in order to use your software.  HardwareSerial is in the Arduino core in a fundamental way since it uses the serial ISRs.

I don't see a clean way to use a #define in a sketch to override the size of the buffers in the HardwareSerial library.

Show me a working example of how to change the size of the ISR receive buffer in the library by using a #define in the user's sketch.

I would like to avoid dynamic memory but simple ideas using a #define don't work.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 170
Posts: 12480
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I don't see a clean way to use a #define in a sketch to override the size of the buffers in the HardwareSerial library.
I don't even see an ugly way smiley-wink
pity ...
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1481
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I thought more and your right, my ugly way won't work.

The fact that HardwareSerial.h is included in almost every library via WProgram.h or Arduino.h in 1.0 killed my ugly idea.
Logged

Valencia, Spain
Offline Offline
Faraday Member
**
Karma: 118
Posts: 4569
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I don't see a clean way to use a #define in a sketch to override the size of the buffers in the HardwareSerial library.

Show me a working example of how to change the size of the ISR receive buffer in the library by using a #define in the user's sketch.

There isn't one, that's why this is in the "suggestions" area...!
Logged

No, I don't answer questions sent in private messages...

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1481
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Why suggest it if it can't be done with a #define.

How would you modify the library to make a #define work?

I have written dozens of Arduino libraries and don't know I would use a #define in a user sketch to set the size of an ISR receive buffer in a library.
Logged

Valencia, Spain
Offline Offline
Faraday Member
**
Karma: 118
Posts: 4569
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

How would you modify the library to make a #define work?

I have written dozens of Arduino libraries and don't know I would use a #define in a user sketch to set the size of an ISR receive buffer in a library.

Like this:

Code:
#ifndef RX_BUFFER_SIZE
#define RX_BUFFER_SIZE 128
#endif

Then the user can do:
Code:
#define RX_BUFFER_SIZE 16
#include "serial.h"
...


Logged

No, I don't answer questions sent in private messages...

Pages: [1] 2   Go Up
Jump to: