Character limit?

Jassper · April 26, 2012, 1:45pm

PaulS:
We need to see your current code. Serial data arrives in it's own time. You have to remove it from the buffer faster than it is placed in the buffer, or you will begin loosing data. delay()s and Serial.print()s cause it to take longer to read all the data.

This is what I am currently working with. It is working but I don't feel confidant that it is correct.

// ****************************************************************
// File Name: 
// Version:
// Created By:
// Comments:
// ****************************************************************

// ********************** Version x.xx ****************************
// 03.11.12

// ********************** Consants ********************************
#define EOP        ';'
#define SERSIZ     25

// ********************** Included Libraries **********************
//#include <SoftwareSerial.h>

// ********************** Defaults ********************************
// ********************** Analog Input Pins ***********************
// ********************** PWM Output Pins *************************
// ********************** Digital Input Pins **********************
// ********************** Digital Output Pins *********************
// ********************** Output Pins *****************************
// ********************** Sensor Pins *****************************

// ********************** Variables *******************************
int i;
int c;
int Percent[4];
int iData[SERSIZ];
int iTemp = 0;

boolean PrintFlag = false;

int index = 0;

// ********************** Other Information ***********************

// ********************** Declorations ****************************
//SoftwareSerial LCDserial(7,8);

// ********************** SET UP **********************************
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  if(Serial.available())
    ReadData();
      
  if(PrintFlag)
    {
    for(i=0; i < SERSIZ; i++)
      {
      Serial.println(iData[i]);
      }
    PrintFlag = false;  
    }
}

// *********************** FUNCTIONS *******************************
// Read Incoming Data from Tank monitor
void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  int count = 0;
  
  while(c != EOP)
    {
    if(count >= 30000)
      {
      Serial.println("end");
      break;
      }
    c = Serial.read();
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      iTemp = 0;
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      PrintFlag = true;
      break; 
      }
    count++;
    }
}
// End ReadData()

system · April 26, 2012, 1:56pm

Counting the number of times that the while loop has iterated is a lousy indication of time.

    if(c > 47 && c < 58)

would make a lot more sense as

    if(c >= '0' && c <= '9')

with no need to consult an ASCII table that the compiler intimately knows.

      iTemp = iTemp * 10 + (c - 48);

c - '0'...

    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      PrintFlag = true;
      break; 
      }

You should NOT be incrementing index here. You've stored the last valid value.

  if(PrintFlag)
    {
    for(i=0; i < SERSIZ; i++)
      {
      Serial.println(iData[i]);
      }
    PrintFlag = false;  
    }

You should only be printing index values, here. You know how many you received. After using/printing the values, some initialization would be a good idea. Wiping the values from iData, for instance. iData is a lousy name, by the way. It gives no clue what is in the array.

system · April 26, 2012, 2:02pm

iData is a lousy name, by the way. It gives no clue what is in the array.

...and could get you into trouble with Apple.

Jassper · April 26, 2012, 2:21pm

PaulS,
Thanks for the clean up tips, but that has no effect on how the code functions. I can still re-create all my issues.

I'm not understanding something , or the Serial.read() don't work the way it is stated. Take this code for example,

int i;
int c;
int iTemp = 0;

boolean PrintFlag = false;

int index = 0;

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  if(Serial.available())
    {
    ReadData();
    }
}

// *********************** FUNCTIONS *******************************
void ReadData()
{
  c = 0;
   
  while(c != ';')
    {
    c = Serial.read();
    if(c == -1)// no data
      {
      Serial.println("end");
      break;
      }
    Serial.println(c);
    }
}
// End ReadData()

If I send, 123 I should get back ,
49
50
51
end

But I don't, I get this instead,
49
end
50
end
51
end

This indicates that every other Serial.read() results in a -1 null character instead of reading the next character.

There is serial available so the function is called, c is loaded with Serial.read() and it is printed as 49 (char '1'), the while loop is no over so c is read again, but this time it is -1 instead of the next character in line to be read ??? There is still data avail so the main loop calls ReadData() again, then 50 is printed, but again the next character read is -1, so on.

Clearly I am missing something.

BTW, what name would you give a bunch of numbers other than "Data"?

EDIT
Further more, if I change it to simply,
void ReadData()
{
c = 0;

while(c != ';')
{
c = Serial.read();
Serial.println(c);
}
}
Sending 123 to the port I get

49
-1
-1
-1
-1
-1
-1
50
-1
-1
-1
-1
-1
51
-1
-1
-1
44
-1
-1
-1
-1

Indicating that the while loop is executing much faster than the data is coming in.

system · April 26, 2012, 2:26pm

But I don't, I get this instead,
49
end
50
end
51
end

That's precisely what I'd expect you to get.
Serial comms are SLOW, you can whip around your while loop loads of times in the time it takes to receive one character (in fact, it's only your serial prints that are slowing you down)

This indicates that every other Serial.read() results in a -1 null character instead of reading the next character.

No, Serial.read returns -1 when there is nothing there to read, i.e. nothing has been received. (-1 is most definitely NOT null)

Follow your code through:
In loop, suddenly "available" returns 1 because a character has just been received.
Call ReadData, set c to zero, test to see if c is a semicolon (it isn't), so read the character that just came in.
it isn't -1, so print it.
Go back to the top of the while, see if the character was a semicolon.
It wasn't a semicolon, so read the the next character.
Oh dear. It hasn't arrived yet.

Jassper · April 26, 2012, 2:31pm

AWOL:

But I don't, I get this instead,
49
end
50
end
51
end

That's precisely whaty I'd expect you o get.
Serial comms are SLOW, you can whip around your while loop loads of times in the time it takes to receive one character (in fact, it's only your serial prints that are slowing you down)

See, I thought that from the very beginning, So I removed the serial printing and made it only print when nothing else was going on just to verify the data. all I got back was 0's as if nothing had been stored. I could only get the data to store if I printed it right away. So I need to verify that there is actually a character present before reading that character.

Jassper · April 26, 2012, 2:46pm

AWOL:
Follow your code through:
In loop, suddenly "available" returns 1 because a character has just been received.
Call ReadData, set c to zero, test to see if c is a semicolon (it isn't), so read the character that just came in.
it isn't -1, so print it.
Go back to the top of the while, see if the character was a semicolon.
It wasn't a semicolon, so read the the next character.
Oh dear. It hasn't arrived yet.

I am seeing that now, the Serial port is very slow which explains why when I add Delays I can get it to work - but that's not the right way to do it.

I guess I should have just asked the question,
How do I separate a string of data? For example I will be receiving something like this,
345,234,532,231,426,63,312,4,523,543;
How do I tun it into
Data[0] = 345
Data[1] = 234
Data[2] = 532
etc etc
?

If I use while(Serial.available()) the wile loop exits before the next character come in. if I check for a ending character ';' I could get stuck in a loop. I agree that counting the loops is a cheesy way of doing it, guess I will half to compare mils to make sure it's not taking too long.

Oh, by the way everyone, I really appreciate the help and head knocking

system · April 26, 2012, 2:53pm

I agree that counting the loops is a cheesy way of doing it, guess I will half to compare mils to make sure it's not taking too long.

It's really very simple:

void ReadData()
{
   unsigned long startTime = millis();
   while(c != ';' && millis() - startTime < 30000)
   {
      // code you currently have
   }

   // At this point, you either got the ; or waited 30 seconds for it, but it didn't arrive
}

system · April 26, 2012, 3:04pm

Why don't you just check one character each time?

When you check serial.available, you don't know how many characters are in the buffer, you just know there are one or more characters.

Maybe this simple example can help you:

#define EOP        ';'
#define SERSIZ     100

int iData[SERSIZ];
int index = 0;
boolean flagPrint = FALSE;

void loop()
{
  int i;
  ///////////////////////// TO READ SERIAL ////////////////////////
  ReadData();

  ///////////////////////// TO CHECK BUFFER ////////////////////////
  if(flagPrint)
  {
    for(i=0;i<index;i++)
      Serial.println(iData[i]);
    index = 0;
    flagPrint = FALSE;
  }
}

// *********************** FUNCTIONS *******************************
void ReadData()
{
  int c = 0;
  
  if(Serial.available())
  {
    c = Serial.read();
    if(c==EOP)  // End of message
      flagPrint = TRUE;
    else
    {
      iData[index++] = c; // Here you can add more actions with the character
    }
  }
}
// End ReadData()

You don't need to add a timer or something like that, because you only read once after serial.available(), if there are more characters in the serial buffer, you will read the next one in the next loop round.

jraskell · April 26, 2012, 3:06pm

Have a look at this: JHaskell's Blog: Serial Comm Fundamentals on Arduino

Then this: JHaskell's Blog

graynomad · April 26, 2012, 3:08pm

Maybe this simple example can help you:

Which is essentially what I posted a while back, I don't know if Jassper tried it or not.

Rob

Jassper · April 26, 2012, 3:48pm

jorgepl:
Why don't you just check one character each time?

That won't give me what I Need. If the first number in is 234 I need Data[0] to be 234, so after reading the 2, I must wait until the 3 and then the 4 come in as well.

The main loop is then going to do something with all this data so it all has to be in, then the rest of the loop. Also, the data will be coming in every second. so I need to repeat the data and do something with it before the next line comes in.

This program is only so I can verify that I am reading the data correctly.

I think I have it working now, thanks to everyone. I never realized that the Serial port was that slow.

system · April 26, 2012, 3:55pm

I never realized that the Serial port was that slow.

  Serial.begin(9600);

You could make it a lot faster.

Jassper · April 26, 2012, 5:07pm

dxw00d:

I never realized that the Serial port was that slow.
  Serial.begin(9600);
You could make it a lot faster.

I realize that, but the data set is coming from something that is sending it at 9600. I will however look into if I can speed that up, but the distance is about 50 feet unshielded so not sure how much faster I want/can push it without possible corruption.

This is what I have now,

// ****************************************************************
// File Name: 
// Version:
// Created By:
// Comments:
// ****************************************************************

// ********************** Version x.xx ****************************
// 03.11.12

// ********************** Consants ********************************
#define EOP        ';'
#define SERSIZ     25

// ********************** Included Libraries **********************
//#include <SoftwareSerial.h>

// ********************** Defaults ********************************
// ********************** Analog Input Pins ***********************
// ********************** PWM Output Pins *************************
// ********************** Digital Input Pins **********************
// ********************** Digital Output Pins *********************
// ********************** Output Pins *****************************
// ********************** Sensor Pins *****************************

// ********************** Variables *******************************
int i;
int c;
int Percent[4];
int iData[SERSIZ];
int iTemp = 0;

boolean PrintFlag = false;

int index = 0;

unsigned long startTime;

// ********************** Other Information ***********************

// ********************** Declorations ****************************
//SoftwareSerial LCDserial(7,8);

// ********************** SET UP **********************************
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  if(Serial.available())
    {
    ReadData();
    }
  
  if(PrintFlag)
    {
    for(i=0; i < SERSIZ; i++)
      {
      Serial.println(iData[i]);
      }
    PrintFlag = false;  
    }
}

// *********************** FUNCTIONS *******************************
// Read Incoming Data from Tank monitor
void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  startTime = millis();
  
  while(c != EOP && millis() - startTime < 600) // Exit if EOP is not seen in less than 0.6 sec
    {
    c = Serial.read();
    if(c >= '0' && c <= '9')
      {
      iTemp = iTemp * 10 + (c - '0');
      }
    else if(c == ',') // Record Data and move to next index
      {
      iData[index] = iTemp;
      index++;
      iTemp = 0;
      }
    else if(c == EOP) // Record data and exit loop
      {
      iData[index] = iTemp;
      PrintFlag = true;
      break; 
      }
    }
}
// End ReadData()

And it appears to be working correctly. It will take incoming data like this;
244,23.5,100,23.6,456,345; (actual string is 24 numbers separated by commas ending in ; )
and record them as
iData[0] 244
iData[1] 235
iData[2] 100
iData[3] 236
etc

Now I can go back to work on what I am doing with this data in the main loop - which isn't much so a lot of time spent just trying to understand what the heck was going on. Gerrr. All well, what doesn't cause me to kill myself makes me stronger right? lol

Thanks everyone and if anyone sees anything else I can improve on - please feel free to point it out.

Topic		Replies	Views
Stringa incompleta Software	27	8799	May 7, 2021
reading serial port Programming	22	5596	May 5, 2021
Long serial communication Programming	27	3310	May 5, 2021
Cant get more than 128 Characters over serial port Syntax & Programs	24	13010	May 6, 2021
Losing data when reading from the Serial port Programming	15	11849	May 5, 2021

Character limit?

Related topics