What is the Arduino Uno Serial buffer size?

I am reading information from my motorcycle's ECM via serial connection on the Arduino Uno R3. I am supposed to read a 107 byte series, but I am only getting the first 62 bytes. The header in the series says there are supposed to be 100 bytes of info (plus 7 for the header) so I know the ECM is trying to send all of it.

Is the buffer size only 62 bytes? How can I get around this so I can read all 107 bytes and save them into an array?

Is the buffer size only 62 bytes?

The buffer size is 64 bytes.

How can I get around this so I can read all 107 bytes and save them into an array?

Generally, the thing to do is read as fast as it arrives. Without seeing your code, we have no idea why you aren't doing this. There is nothing to get around, except to reading before the data is lost.

What is the baud-rate of the serial data you are receiving? That will have a bearing of how fast you must read the data from the serial buffer into your own user defined buffer of the size required to hold the message. But bottom line you should not be reliant on the size of the serial buffer, but rather set a baud-rate such that you can transfer characters fast enough into your own buffer such that no serial buffer over-runs occurs.

Lefty

The baud rate that the ECM is 9600 sending and receiving, so I am using 9600 for the Arduino as well. Hmmm, maybe the delay I am using is causing part of the problem, but that doesn't help if the buffer is only 64 bytes. I wonder why I am receiving 62 bytes instead...

Here is my code I am using:

/*  
 This program is used for an Arduino to request and retrieve
 data from an 07 Buell XB9SX Lightning ECM DDFI-2,
 EEPROM Version: BUEIB
 
 Currently the Arduino successfully sends a request code and
 receives only 62 bytes of the run-time response, and displays
 on the LCD screen and via serial monitor.
 
 Created by Michael Blaylock
 */

// include the library code:
#include <LiquidCrystal.h>
#include <SoftwareSerial.h>

//new serial pins for ECM, 0 and 1 caused interferance from PC
#define rxPin 2
#define txPin 3

// set up a new serial port
SoftwareSerial mySerial =  SoftwareSerial(rxPin, txPin);

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 8, 9, 10, 11, 12);
byte inArray[9]; //request code for real-time data
byte outArray[107]; //real-time data series from ECM
byte long Value;
int First = 0;

void setup(){
  // set up the LCD's number of columns and rows: 
  lcd.begin(16, 2);
  // initialize the serial communications:
  // define pin modes for tx, rx:
  pinMode(rxPin, INPUT);
  pinMode(txPin, OUTPUT);
  // baud rate for the ECM is 9600
  mySerial.begin(9600);
  Serial.begin(9600);
  inArray[0]=0x01; //SOH
  inArray[1]=0x00; //Emittend
  inArray[2]=0x42; //Recipient
  inArray[3]=0x02; //Data Size
  inArray[4]=0xFF; //EOH
  inArray[5]=0x02; //SOT
  inArray[6]=0x43; //Data 1 //0x56 = Get version, 0x43 = Get runttime data
  inArray[7]=0x03; //EOT
  inArray[8]=0xFD; //Checksum
  //Stream request data series to ECM as HEX
}

//print only 1 response to serial monitor to avoid flooding
void display() {
  if(First == 0) {
    for (int i=0; i<107; i++){
      Serial.println(outArray[i],HEX);
    }
    First = 1;
  }
}

void loop() {
  lcd.setCursor(0,0);
  for (int i = 0; i<9; i+=1){
    mySerial.write(inArray[i]);
    lcd.print(inArray[i],HEX); //print sent HEX code
  }
  // when characters arrive over the serial port...
  if (mySerial.available()) {
    // wait a bit for the entire message to arrive
    delay(100);
    lcd.setCursor(0, 1);
    // read all the available characters
    while (mySerial.available() > 0) {
      // listen for new serial coming in:
      for (int i=0; i<107; i++){
        outArray[i] = mySerial.read();
      }
      //save specific bytes as useful variables
      //concatenate 2 byte RPM data
      unsigned Value = outArray[31] << 8 | outArray[30];
      //byte Value = outArray[90];
      lcd.setCursor(0,1);
      lcd.print("temp: ");
      lcd.print((Value*0.18-40)/2);
      //lcd.print(" ");
      //lcd.print(outArray[28],HEX);
      //lcd.print(outArray[29],HEX);
      display();  
    }
  }
}
        outArray = mySerial.read();

outArray is not an element of an array.

Should I have:

outArray[i] = mySerial.read();?

I found the formatting I have been using from another thread on the forums.

      for (int i=0; i<107; i++){
        outArray = mySerial.read();
      }

Given the location of that line, it would make sense.

Hmmm, maybe the delay I am using is causing part of the problem

It doesn't help. But look at this:

    while (mySerial.available() > 0) {
      // listen for new serial coming in:
      for (int i=0; i<107; i++){
        outArray = mySerial.read();
      }

Suppose 8 bytes have arrived. The while condition will be true, so you read all 107 of the 8 bytes. Does that really seem like a good idea?

#define rxPin 2
#define txPin 3

Good.

SoftwareSerial mySerial =  SoftwareSerial(rxPin, txPin);

Give complete control of the pins to SoftwareSerial.

  pinMode(rxPin, INPUT);
  pinMode(txPin, OUTPUT);

Then, muck with them, anyway. Not a good idea.

It's a side effect of not using CODE TAGS when posting code.
Note the italics that appeared.

I missed that. The whole thing is a quote, so it's all in italics.

dxw00d:

      for (int i=0; i<107; i++){

outArray = mySerial.read();
      }




Given the location of that line, it would make sense.

I fixed it as code in my post.

I was using the same code and was sending the wrong checksum so the ECM was sending me a 10 byte error. It was reading that fine but the rest of the outArray was empty with 0xFF.

So I guess what is happening is it is writing mySerial.read() to outArray 107 times? If I tell it to write mySerial.read() to outArray i location, wouldn't it be trying to write the entire buffer to that one location, or will it write the very first byte in the buffer to the i location in the outArray?

wouldn't it be trying to write the entire buffer to that one location

Time to go read the documentation on the Serial.read() function. It does not read "the entire buffer" at once. It reads one byte at a time. So, it would only write one byte to outArra[ i ].

Of course, that isn't you real problem.

ok so doing outArray[ i ] = mySerial.read() should fix my problem?

It sounds like I was unintentionally writing the entire buffer at once, when I should be writing the first incoming byte of the buffer to outArray[ i ].

The ECM is supposed to send out 107 bytes every time unless it is sending out an error message.

I changed it to outArray[ i ] = mySerial.read() and the problem still persists.

I tested it and it says that the buffer is overflowing.

It looks like when you use the mySerial.read() function, it doesn't clear out the previous values that were read, so the buffer fills up to 64 and no more values come in.

How can I clear each byte that has already been read from the buffer so that It does not overflow?

EDIT: So I rearragned the location of the mySerial.read() and some other things and I was able to read 67 of the 107 bytes. Am I just not reading fast enough? I am using the correct baud for my ECM. I'll post the revised code, I just need to clean it up so you guys can make sense of it.

Blaylock1988:
Is the buffer size only 62 bytes? How can I get around this so I can read all 107 bytes and save them into an array?

A suitable method is described here:

// wait a bit for the entire message to arrive
    delay(100);

You don't need any delays if you follow my method.

I tried using your method with the processIncomingByte stuff, but unfortunately I still only received 67 bytes.

Here is my current code:

/*  
 This program is used for an Arduino to request and retrieve
 data from an 07 Buell XB9SX Lightning ECM DDFI-2,
 EEPROM Version: BUEIB
 
 Currently the Arduino successfully sends a request code and
 receives only 67 bytes of the run-time response, and displays
 on the LCD screen and via serial monitor.
 
 Created by Michael Blaylock
 */

// include the library code:
#include <LiquidCrystal.h>
#include <SoftwareSerial.h>

//new serial pins for ECM, 0 and 1 caused interferance from PC
#define rxPin 2
#define txPin 3

// set up a new serial port
SoftwareSerial mySerial =  SoftwareSerial(rxPin, txPin);

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 8, 9, 10, 11, 12);
byte inArray[9]; //request code for real-time data
byte outArray[107]; //real-time data series from ECM
byte long Value;
int First = 0;
int q;

void setup(){

  lcd.begin(16, 2);  // set up the LCD's number of columns and rows: 


  mySerial.begin(9600);  // baud rate for the ECM is 9600
  Serial.begin(9600);
  q=0;
  inArray[0]=0x01; //SOH
  inArray[1]=0x00; //Emittend
  inArray[2]=0x42; //Recipient
  inArray[3]=0x02; //Data Size
  inArray[4]=0xFF; //EOH
  inArray[5]=0x02; //SOT
  inArray[6]=0x43; //Data 1 //0x56 = Get version, 0x43 = Get runttime data
  inArray[7]=0x03; //EOT
  inArray[8]=0xFD; //Checksum

  lcd.setCursor(0,0);
  //Stream request data series to ECM as HEX
  for (int i = 0; i<9; i+=1){
    mySerial.write(inArray[i]);
    lcd.print(inArray[i],HEX); //print sent HEX code
  }
}

void processIncomingByte (const byte c){
  q++;
  outArray[q] = c;
  Serial.println(outArray[q],HEX);
  if(q > 107){
    q = 0;
  }
}

void loop() {
  //read the incoming data
  if (mySerial.available())
  processIncomingByte (mySerial.read());

  //concatenate 2 byte RPM data
  //    unsigned Value = outArray[31] << 8 | outArray[30];
  //    lcd.setCursor(0,1);
  //    lcd.print("temp: ");
  //    lcd.print((Value*0.18-40));
  //    display();  
}

Yes, well I was thinking more like this:

/*  
 This program is used for an Arduino to request and retrieve
 data from an 07 Buell XB9SX Lightning ECM DDFI-2,
 EEPROM Version: BUEIB
 
 Currently the Arduino successfully sends a request code and
 receives only 67 bytes of the run-time response, and displays
 on the LCD screen and via serial monitor.
 
 Created by Michael Blaylock
 */

// include the library code:
#include <LiquidCrystal.h>
#include <SoftwareSerial.h>

//new serial pins for ECM, 0 and 1 caused interferance from PC
#define rxPin 2
#define txPin 3

// set up a new serial port
SoftwareSerial mySerial =  SoftwareSerial(rxPin, txPin);

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 8, 9, 10, 11, 12);
const byte inArray[9] = {
  0x01, //SOH
  0x00, //Emittend
  0x42, //Recipient
  0x02, //Data Size
  0xFF, //EOH
  0x02, //SOT
  0x43, //Data 1 //0x56 = Get version, 0x43 = Get runttime data
  0x03, //EOT
  0xFD, //Checksum

} 
; //request code for real-time data
byte outArray[107]; //real-time data series from ECM
int First = 0;
int pos;

void setup(){

  lcd.begin(16, 2);  // set up the LCD's number of columns and rows: 

  mySerial.begin(9600);  // baud rate for the ECM is 9600
  Serial.begin(9600);
  pos=0;

  lcd.setCursor(0,0);
  //Stream request data series to ECM as HEX
  for (int i = 0; i<9; i+=1){
    mySerial.write(inArray[i]);
    lcd.print(inArray[i],HEX); //print sent HEX code
  }
}  // end of setup


void loop() 
{
  // fill buffer
  if (Serial.available () > 0) 
    outArray[pos++] = Serial.read ();

  if (pos >= 107)
  {
    // process it

    pos = 0;  // ready for next time  

  }  // end if full
}  // end of loop

Thank you very much for the help. I will give your version a shot, but I tried something very similar to it and I was getting overflows in the arduino buffer.

Where you commented "buffer" were you referring to the arduino buffer or the outArray?

outArray.

The code in loop is pulling data from the Arduino internal buffer (which has a fixed size) into your buffer (which is the size you want) as fast as it can. Then when you have the number of bytes you want, you process them.

Are you certain you are getting 107 bytes? Or is it "up to" 107 bytes?

The run-time data that the ECM sends out is a 107 byte series of data.

The first table on the EcmSpy website shows the data that is coming in:
http://www.ecmspy.com/cgi-bin/runtime.cgi

The only time that the ECM will be sending out anything smaller is if there is an error code (which is around 10 bytes) but this only happens when I send an incorrect request or if I crash the bike.