Pages: [1]   Go Down
Author Topic: Serial.read for high data rate input  (Read 941 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi,

I'm using an Arduino Mega 2560 to handle the data of a Sparkfun Razor 9DOF IMU, which is constantly sending the accelerometers, gyroscopes and magnetometers values through a Serial connection to the Arduino. However, I'm loosing several data when reading the raw output of the IMU in the Arduino. This only happens when I add more code besides the typical output of the Serial1 input. Seems like the Arduino doesn't have a consistent buffer system to temporarily store the data that is being received on the serial port when it is executing other code.

Example:
This produces the correct output:
Code:
if(Serial1.available() > 0)
  Serial.print(Serial1.read(), BYTE);

This (most of the times) doesn't produce the correct output:
Code:
if(Serial1.available() > 0) {
    char ch = Serial1.read();
    
    if (ch == '\n') {
       Serial.println(s);
      //some parser processing
      s = "";
    }
    else s += ch;
}

I have created a workaround by flushing and discarding some characters in order to keep receiving valid data (correct sentences) but that significantly reduces the update rate of the IMU. Have you ever faced this problem before? and do you know any other solution for it?

Thanks in advance.
Regards,
André.
« Last Edit: March 31, 2011, 05:31:03 pm by andresp » Logged

Manchester (England England)
Offline Offline
Brattain Member
*****
Karma: 505
Posts: 31332
Solder is electric glue
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Don't print out the data you receive, this is robbing you of the time.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I need to print the data to the Serial port to monitor it (otherwise I wouldn't know whether it is working correctly or not). Even if it was not for monitoring purposes I would still need to print data to the Serial port (at basically the same data rate) because it is used in a Java application.
Logged

Manchester (England England)
Offline Offline
Brattain Member
*****
Karma: 505
Posts: 31332
Solder is electric glue
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Then you are stuffed.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

smiley-roll-sweat
I think I will need to process the data in the Java application then. thanks anyway  smiley-wink
Logged

0
Offline Offline
God Member
*****
Karma: 0
Posts: 594
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Arduino's serial receiving code uses a FIFO buffer, but the transmit function is blocking (waits until transmission is finished). You can implement your own buffered transmit code if you want more parallelism
Logged

Freelance engineer, consultant, contractor. Graduated from UW in 2013.

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Actually I think the problem is more related with some problem with that FIFO reader while executing (a little bit more intensive) code (maybe the input from the IMU exceeds Arduino's serial buffer size?) than with the blocking printing, because if I remove the parser processing and keep all the remaining code the problem disappears.

Like this:
Code:
 if (Serial1.available() > 0) {
    char ch = Serial1.read();
    
    if (ch == '\n') {
       Serial.println(s);
      //without any parser processing
      s = "";
    }
    else s += ch;
  }

I think that if the problem was being caused by the blocking print, it would still occur in this situation. However, that is not the case.
« Last Edit: March 31, 2011, 06:51:43 pm by andresp » Logged

0
Offline Offline
Tesla Member
***
Karma: 114
Posts: 8894
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
if I remove the parser processing and keep all the remaining code the problem disappears

Sounds like you need better parser code.
Logged

Consider the daffodil. And while you're doing that, I'll be over here, looking through your stuff.   smiley-cool

Left Coast, CA (USA)
Offline Offline
Brattain Member
*****
Karma: 331
Posts: 16459
Measurement changes behavior
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Well a quick test might be useful. Try and increase the receiver buffer size used for the hardware serial library code. The file name is HardwareSerial.cpp and it's located in the arduino core folders (in my machine at: C:\Documents and Settings\Primary Windows User\My Documents\My Programs\Arduino\arduino-0022\hardware\arduino\cores\arduino )

Here is the section that defines how large the serial receiver buffers are:

Quote
// Define constants and variables for buffering incoming serial data.  We're
// using a ring buffer (I think), in which rx_buffer_head is the index of the
// location to which to write the next incoming character and rx_buffer_tail
// is the index of the location from which to read.
#if (RAMEND < 1000)
  #define RX_BUFFER_SIZE 32
#else
  #define RX_BUFFER_SIZE 128
#endif

You can try bumping that 128 to 256 or even 512 and see if that helps your symptom.

Lefty
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

That's it Lefty.

It didn't solve the problem (even with 1024 as the buffer's size) but I was able to confirm that the cause is the serial buffer overflow.

Basically as the program processes each line, 20 new chars are added to the buffer (as can be seen by checking out Serial1.available()). This causes the buffer overflow within some iterations (depending on the size of the buffer).

Thanks a lot Lefty.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

By limiting the printing to one third of the iterations the solution becomes sustainable. I will try to process the data using this solution, I think it should be enough if it doesn't require me to further reduce the communication. Thank you all for your quick and useful help.
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 547
Posts: 45979
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You are doing a lot of String object allocation, copying, and deallocation by adding one character at a time to a String object. If you were to allocate and manage a character array, you could eliminate all that activity, and speed up your processing of each incoming character.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 7
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Another useful tip is to increase the baud rate of the Computer - Arduino serial port. By using 115200, I was able to print all the data without any problem
« Last Edit: April 01, 2011, 11:48:50 am by andresp » Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 547
Posts: 45979
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Code:
Another useful tip is to increase the baud rate of the Computer - Arduino serial port. By using 115200, I was able to print all the data without any problem
This is exactly why we often ask people to post all of their code. Since the snippet of code did not include the Seril.begin() call, we simply assumed that you were running it as fast as possible. Since that was not the case, posting all of your code would have allowed us to spot that, and get you to the real solution much sooner.
Logged

Pages: [1]   Go Up
Jump to: