Extra leading "new line" character when reading over Serial

[deyan_i]

Slightly strange problem I can't seem to explain probably because of my lack of knowledge of the architecture. Anyways, here goes.

I am reading long strings through the Serial port. When I read the first string and place it in an array, everything I sent gets received, no problem. However, second string I send, seems to pick up an extra character on the way and when I read back my character array, I get a leading "new line" character.

char input;
char cArray[80];
String commandString;
 
struct g_code
{
  char letter;
  int numb;
  float x_coord;
  float y_coord;
  float z_coord;
  float i_value;
  float j_value;
} command;

void setup(){
  //Setup Communications
  Serial.begin(9600);
}

void loop(){

  //Check for Serial Commands
    if(Serial.available()>0){
      receiveCommand();
      Serial.println("Received");
      processString();
      Serial.println(command.letter);
    }
   
}

void receiveCommand(){
  int i=0;
  do{
    //Read Serial buffer
    input = Serial.read(); 
    if(input != -1) {
      // Add to Array if not equal to -1.
      // -1 means there is no data to be read over
      // Serial right now.
      cArray[ i] = input;
      i++;
    }
  }while(input != '\r');
  Serial.flush();
}

void processString(){
    for(int x=0; x<80; x++){
   Serial.print(cArray[ x],DEC);
  }
  Serial.println();
 
  command.letter = cArray[0];
 
}

and these are my results over the Serial monitor, notice the leading 10 which is ASCII for new line.

Received
714850130000000000000000000000000000000000000000000000000000000000000000000000000000
G
Received
1071485013000000000000000000000000000000000000000000000000000000000000000000000000000


Any thoughts on this? I've noticed that if I add some delays ~50ms or more between the Serial.available and Serial.read functions, this doesn't seem to happen. Timing issue?

[Grumpy_Mike]

When posting code use the # icon not the quote on next to it.

You do:-
if(Serial.available()>0){
      receiveCommand();

but then in the receiveCommand() function you carry on reading data that might not have arrived in the buffer. Before you read each byte you need to halt or wait until  Serial.available()>0, then only when you receive '\r' do you stop looking.

Are you sure that your sending application is not sending a CR and LF?

[deyan_i]

Thanks for the heads up Grumpy_Mike, I modified my code and tried this:

Code: [Select]

void receiveCommand(){
  int i=0;
  do{
    if(Serial.available()>0){
      input = Serial.read();
      cArray[i]=input;
      i++;
    }
  }while(input != '\r');
  Serial.flush();
}

But unfortunately, it did not work, I am still getting the leading new line character on the second string. As for my input method, I am just pasting characters from a txt file into the Arduino IDE Serial Monitor.
I was under the impression that if the Serial.available command returned a value greater than 0 then there is stuff available to read in the buffer, so you need not call it again before the first read. Am I wrong?

I should also mention that with this code modification the error only occurs when I send long strings, such as:
G03 X104.926701 Y95.976951 Z-0.125000 I-0.840857 J-3.068969 F400.000000

[PaulS]

Code: [Select]

  Serial.flush();
Why? If the sending program sends a second packet before the Arduino has read the end of the first packet, random amounts of data will be thrown away.

If you don't want the sender to send more data before the first packet has been received, read, and processed, you need to implement handshaking. If the sender CAN send another packet before the first one has been completely read, then you must get rid of the flush(), or be prepared to deal with incomplete packets (that YOU have made incomplete by throwing away random amounts of data).

Code: [Select]

  do{
    if(Serial.available()>0){
      input = Serial.read();
      cArray[i]=input;
      i++;
    }
  }while(input != '\r');
A do/while loop body will be executed at least once, before the test is performed. It is a very rarely used structure, as a result. I recommend that you not use it, either. Use a standard while loop, with proper initialization of the control variable.

When you do, you will see that you have already added the character in input to the end of the array before you test what it's value is.

You are not properly NULL terminating the array, either.

When printing stuff, just jamming numbers out to the serial monitor doesn't cut it. Some identifier of the data is needed, or white space between consecutive values, and blank lines between each execution of Serial.println().

Finally, what is sending the data to the Arduino? Typically, if that program is a Windows application, the carriage return gets translated to a carriage return, line feed combination. You are testing for one to stop the loop, but not ignoring the other.

Before adding input to the array, make sure it is not a carriage return and is not a line feed.

Code: [Select]

if(input != '\n' && input != '\r')
{
  cArray[i++] = input;
  cArray[i] = '\0';
}

[deyan_i]

As I mentioned earlier, I am using the Arduino IDE Serial Monitor to send strings over Serial.

Thank you for the array terminator reminder, and I apologize for the random Serial print outs, they just help me debug. This is just a stripped down version of my code to isolate the problem, I swear it all makes sense when I put it together.

Your method seems to work PaulS, because I am now filtering for \n and \r characters, but that doesn't really explain why that \n character should be there in the first place if I never sent it...

The reason I had the Serial.flush function was to make sure that I cleared the buffer of any extra characters that may have come in with the Windows carriage return at the end of the string ie. \r\n. That however doesn't seem to work either,the only logical explanation I can come up with is that the \n character gets left behind in the Serial buffer so that the next time there is data to read it is the first character in line to pop out.

[PaulS]

Your method seems to work PaulS, because I am now filtering for \n and \r characters, but that doesn't really explain why that \n character should be there in the first place if I never sent it...

Maybe you did.

As I mentioned earlier, I am using the Arduino IDE Serial Monitor to send strings over Serial.

With what option (at the bottom of the Serial Monitor) selected?

The reason I had the Serial.flush function was to make sure that I cleared the buffer of any extra characters that may have come in with the Windows carriage return at the end of the string ie. \r\n. That however doesn't seem to work either,the only logical explanation I can come up with is that the \n character gets left behind in the Serial buffer so that the next time there is data to read it is the first character in line to pop out.

The Serial.flush() probably completed (dumping an empty buffer) before the \n arrived.

[bperrybap]

Paul,
I've recently (just yesterday) saw this as well (on 0022) while I was doing some GLCD testing.
It was a simple loop of reading the serial data and dumping it to the GLCD library code for display.
I was using the IDE serial monitor and your GLCD backplane with a Teensy++ 2.0

It is definitely an extra newline character showing up and it doesn't happen all the time
even on the same output.

I was finally able to track it down and it is so stupidly silly.
In my case, my mouse is dieing and occasionally it sends two left clicks rather than one.
So if I used the [send] button on the IDE terminal monitor vs the [enter] on the keyboard,
it occasional would double click
the [send], this will send any data in the text box and then an extra newline.

--- bill

[PaulS]

In my case, my mouse is dieing

Mice are cheap. Bury the existing one soon.

this will send any data in the text box and then an extra newline.

It will send the text in the box, followed by a new line, followed by the text in the box (now none) and a new line - not quite the same thing, but the same result.

[bperrybap]

this will send any data in the text box and then an extra newline.

It will send the text in the box, followed by a new line, followed by the text in the box (now none) and a new line - not quite the same thing, but the same result.

Ok, ok... That's what I meant. I know how it works.
But the apparent "extra" data can vary depending on the line ending option selected in the IDE terminal monitor.
It might be nothing if you have none, or an extra nl, or cr or both.

--- bill