Character limit?

Yep, was going to post my code but was on my Android at the time. (btw, I'm looking for a Arduino file viewer/editor for Android :slight_smile: )

Anyways, I'm sure there is a better way to read incoming serial than I have, but after some testing everything works until I reach 64 characters.

I stripped everything out just to make sure it wasn't anything else i was doing.

// ****************************************************************
// File Name: 
// Version:
// Created By:
// Comments:
// ****************************************************************

// ********************** Version x.xx ****************************
// 03.11.12

// ********************** Consants ********************************
#define EOP        ';'
#define SERSIZ     25

// ********************** Included Libraries **********************
//#include <SoftwareSerial.h>

// ********************** Defaults ********************************
// ********************** Analog Input Pins ***********************
// ********************** PWM Output Pins *************************
// ********************** Digital Input Pins **********************
// ********************** Digital Output Pins *********************
// ********************** Output Pins *****************************
// ********************** Sensor Pins *****************************

// ********************** Variables *******************************
int i;
int c;
int Percent[4];
int iData[SERSIZ];
int iTemp;

byte index = 0;

// ********************** Other Information ***********************

// ********************** Declorations ****************************
//SoftwareSerial LCDserial(7,8);

// ********************** SET UP **********************************
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  ReadData();
    
  for(i=0; i < SERSIZ; i++)
    {
    Serial.println(iData[i]);
    }
  delay(1000);
    
}

// *********************** FUNCTIONS *******************************
// Read Incoming Data from Tank monitor
void ReadData()
{
  while(Serial.available() > 0)
    {
    //char aChar = Serial.read();
    c = Serial.read();
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      iData[index] = '\0';
      iTemp = '\0';
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      iData[index] = '\0';
      break; 
      }
    }
  index=0;
  c = 0;
  iTemp = '\0';
}
// End ReadData()

If I send this, 48 characters,
4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4;

I get back,
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
0 // 25th character is null

But if I send this, 64 characters
13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,13,3,3,3,3,3,3,3,3;

I get this
13
13
13
13
13
13
13
13
13
13
13
13
13
13
13
13
3
3
3
3
3
3
3
0
0

The 24th character becomes null and it gets worse the more over characters I send.

But I'm completely open to it being me :wink:

It looks to me that your code will read the chars in the buffer very quickly, after that you will only get a single char at a time because your ReadData() function does nothing for 99.9% of the time because the Arduino is waiting for characters.

while(Serial.available() > 0)

If there is no chars you return and print something, probably a \0 which you don't see.

After a mS or so a character comes in, you print it then go back to printing \0s because the next char hasn't arrived yet.

Exactly when in the data stream do you want to print? If it's at EOP then

    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      iData[index] = '\0';
      printFlag = true;    // set a flag
      break;
  ReadData();
  if (printFlag ) { 
     for(i=0; i < SERSIZ; i++)
       {
       Serial.println(iData[i]);
       }
   }
   printFlag = false;

also there are quite a few things that code be tidied up, for example this

      iData[index] = iTemp;
      index++;
      iData[index] = '\0';

can be

      iData[index++] = iTemp;
      iData[index] = '\0';

Rob

My idea:

You send those 64 characters while loop() hangs in the delay(1000) function. Thus the last one gets lost.

delay() is bad, at least in your case :wink:

Graynomad's proposal implies that ReadData() is called continuously, and signals when a complete block with ";" has arrived.

michael_x:
My idea:

You send those 64 characters while loop() hangs in the delay(1000) function. Thus the last one gets lost.

delay() is bad, at least in your case :wink:

Graynomad's proposal implies that ReadData() is called continuously, and signals when a complete block with ";" has arrived.

The Delay makes no difference, I thought that too removed it, no different.

Graynomad,
After tiring your suggestions, I get nothing but 0's no matter what I do.
I don't need this to print at all, I was just using that to make sure that the incoming string was getting stored correctly.

Thanks for the tidy tip, I assume then that this
iData[index++] = iTemp;
iData[index] = '\0';

Adds one to index after it assigns iTemp?

I'm doing something really wrong here.

If I add a Serial.println("Serial Available"); within the loop, and send 2,2,2; I will get 6 "Serial Available" and then 2 2 2 and the rest 0's. If I remove that Serial.print line I get nothing but 0's.

???

Graynomad's proposal implies that ReadData() is called continuously,

Yeah I didn't spot the delay(1000).


Rob

Your sticking '\0' into ints, probably OK but just use 0

iData[index] = 0;
iTemp = 0;


Rob

Graynomad:
Your sticking '\0' into ints, probably OK but just use 0

iData[index] = 0;
iTemp = 0;


Rob

Yep tried that too. No different.
I think it is some kind of timing issue or something. If I remove the delay, and only print when EOP is reached, I get all 0's. If I add a deay - either in the main loop or in the function, I will get some results - although not accurate results. If I print every loop I will get results (again not accurate) with or without the delay.

I'm going to try making it read the serail until the EOP is reached instead and see how that works out.

I've changed the function to this,

void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  while(c != EOP)
    {
    c = Serial.read();
    if(c == 0)
      {
      Serial.println("Null found.. Ending ReadData()");
      break;
      }
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      iData[index] = '\0';
      iTemp = 0;
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      iData[index] = '\0';
      PrintFlag = true;
      break; 
      }
    }
}

But how do I exit the while loop if the EOP isn't reached? The if(c == 0) line doesn't work and if I check for Serial.available() it will exit as soon as it is called.

I'll have a look, meanwhile can you try this

// ********************** Consants ********************************
#define EOP        ';'
#define SERSIZ     25


// ********************** Variables *******************************
int Percent[4];
int iData[SERSIZ];
int iTemp = 0;
boolean printData = false;
byte index = 0;

// ********************** SET UP **********************************
void setup()
{
  Serial.begin(9600);
  while(Serial.read() != EOP) {}; // sync with frames
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
	ReadData();
    
	if (printData) {
		for(int i=0; i < SERSIZ; i++) {
			Serial.println(iData[i]);
		}
		printData = false;
	}
 }

// *********************** FUNCTIONS *******************************
// Read Incoming Data from Tank monitor
void ReadData() {

	if(Serial.available() > 0) {
		int c = Serial.read();
		switch (c) {
			case ',':
				iData[index++] = iTemp;
				iData[index] = 0;
				iTemp = 0;
				break;

			case EOP:
				iData[index++] = iTemp;
				iData[index] = 0;
				printData = true;
				iTemp = 0;
				index = 0;
				break;
			
			default:
				if(c >= '0' && c <= '9') {
					iTemp = (iTemp * 10) + (c - '0');  // use () for each part, don't trust operator precedance
				}
		}
	}
}
// End ReadData()

Rob

Ok, I give up. I am totally misunderstanding how the serial thing works.

This works, but could get trapped in a infinite loop if ';' is never reached.

void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  while(c != EOP)
    {
    c = Serial.read();
   
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      //iData[index] = '\0';
      iTemp = 0;
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      //iData[index] = '\0';
      PrintFlag = true;
      break; 
      }
    }
}

If I change it and add a check for c to == -1, it doesn't work.

void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  while(c != EOP)
    {
    c = Serial.read();
    if(c == -1)
      {
      Serial.print("end");
      break;
      }
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      //iData[index] = '\0';
      iTemp = 0;
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      //iData[index] = '\0';
      PrintFlag = true;
      break; 
      }
    }
}

c should be equal to what was read with c = Serial.read(), so if I send a 1,2; c should first equal "1", but it does not, in stead I get "end" and the function exits. In fact I will get "end" 3 times, then 25 0's from the array print.

his works, but could get trapped in a infinite loop if ';' is never reached.

You could put in a timeout

This appears to all be a timing type issue, which makes how the Serial data is handled unreliable. I can get the code working, but I can change the results, or how well it works by adding delay() in different parts of the code, or a simple Serial.print() added here or there will yield different results. It is almost like the data is wiped from the buffer faster than it can be processed or something.

Either I am doing something wrong or there is something wrong with my Arduino. I can't imagine that this issue hasn't happened before because I shouldn't half to adjust how I handle the data based on how many lines of code - thus causing different timings - I have in the program. In otherwords, adding more code shouldn't change my Serial results, but it does.

I'm switching Arduino's and see what results I get.

adding more code shouldn't change my Serial results, but it does.

We need to see your current code. Serial data arrives in it's own time. You have to remove it from the buffer faster than it is placed in the buffer, or you will begin loosing data. delay()s and Serial.print()s cause it to take longer to read all the data.

If you are sending less than a buffer's worth of data, though, you could take all day to read the data, and not risk loosing data.

That you are having such difficulties means that the problem is in your code. Post it again.

PaulS:
We need to see your current code. Serial data arrives in it's own time. You have to remove it from the buffer faster than it is placed in the buffer, or you will begin loosing data. delay()s and Serial.print()s cause it to take longer to read all the data.

This is what I am currently working with. It is working but I don't feel confidant that it is correct.

// ****************************************************************
// File Name: 
// Version:
// Created By:
// Comments:
// ****************************************************************

// ********************** Version x.xx ****************************
// 03.11.12

// ********************** Consants ********************************
#define EOP        ';'
#define SERSIZ     25

// ********************** Included Libraries **********************
//#include <SoftwareSerial.h>

// ********************** Defaults ********************************
// ********************** Analog Input Pins ***********************
// ********************** PWM Output Pins *************************
// ********************** Digital Input Pins **********************
// ********************** Digital Output Pins *********************
// ********************** Output Pins *****************************
// ********************** Sensor Pins *****************************

// ********************** Variables *******************************
int i;
int c;
int Percent[4];
int iData[SERSIZ];
int iTemp = 0;

boolean PrintFlag = false;

int index = 0;

// ********************** Other Information ***********************

// ********************** Declorations ****************************
//SoftwareSerial LCDserial(7,8);

// ********************** SET UP **********************************
void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  if(Serial.available())
    ReadData();
      
  if(PrintFlag)
    {
    for(i=0; i < SERSIZ; i++)
      {
      Serial.println(iData[i]);
      }
    PrintFlag = false;  
    }
}

// *********************** FUNCTIONS *******************************
// Read Incoming Data from Tank monitor
void ReadData()
{
  index = 0;
  iTemp = 0;
  c = 0;
  int count = 0;
  
  while(c != EOP)
    {
    if(count >= 30000)
      {
      Serial.println("end");
      break;
      }
    c = Serial.read();
    if(c > 47 && c < 58)
      {
      iTemp = iTemp * 10 + (c - 48);
      }
    else if(c == ',') // move to next index
      {
      iData[index] = iTemp;
      index++;
      iTemp = 0;
      }
    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      PrintFlag = true;
      break; 
      }
    count++;
    }
}
// End ReadData()

Counting the number of times that the while loop has iterated is a lousy indication of time.

    if(c > 47 && c < 58)

would make a lot more sense as

    if(c >= '0' && c <= '9')

with no need to consult an ASCII table that the compiler intimately knows.

      iTemp = iTemp * 10 + (c - 48);

c - '0'...

    else if(c == EOP)
      {
      iData[index] = iTemp;
      index++;
      PrintFlag = true;
      break; 
      }

You should NOT be incrementing index here. You've stored the last valid value.

  if(PrintFlag)
    {
    for(i=0; i < SERSIZ; i++)
      {
      Serial.println(iData[i]);
      }
    PrintFlag = false;  
    }

You should only be printing index values, here. You know how many you received. After using/printing the values, some initialization would be a good idea. Wiping the values from iData, for instance. iData is a lousy name, by the way. It gives no clue what is in the array.

iData is a lousy name, by the way. It gives no clue what is in the array.

...and could get you into trouble with Apple.

PaulS,
Thanks for the clean up tips, but that has no effect on how the code functions. I can still re-create all my issues.

I'm not understanding something , or the Serial.read() don't work the way it is stated. Take this code for example,

int i;
int c;
int iTemp = 0;

boolean PrintFlag = false;

int index = 0;

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  ///////////////////////// TO READ SERIAL ////////////////////////
  if(Serial.available())
    {
    ReadData();
    }
}

// *********************** FUNCTIONS *******************************
void ReadData()
{
  c = 0;
   
  while(c != ';')
    {
    c = Serial.read();
    if(c == -1)// no data
      {
      Serial.println("end");
      break;
      }
    Serial.println(c);
    }
}
// End ReadData()

If I send, 123 I should get back ,
49
50
51
end

But I don't, I get this instead,
49
end
50
end
51
end

This indicates that every other Serial.read() results in a -1 null character instead of reading the next character.

There is serial available so the function is called, c is loaded with Serial.read() and it is printed as 49 (char '1'), the while loop is no over so c is read again, but this time it is -1 instead of the next character in line to be read ??? There is still data avail so the main loop calls ReadData() again, then 50 is printed, but again the next character read is -1, so on.

Clearly I am missing something.

BTW, what name would you give a bunch of numbers other than "Data"?

EDIT
Further more, if I change it to simply,
void ReadData()
{
c = 0;

while(c != ';')
{
c = Serial.read();
Serial.println(c);
}
}
Sending 123 to the port I get

49
-1
-1
-1
-1
-1
-1
50
-1
-1
-1
-1
-1
51
-1
-1
-1
44
-1
-1
-1
-1

Indicating that the while loop is executing much faster than the data is coming in.

But I don't, I get this instead,
49
end
50
end
51
end

That's precisely what I'd expect you to get.
Serial comms are SLOW, you can whip around your while loop loads of times in the time it takes to receive one character (in fact, it's only your serial prints that are slowing you down)

This indicates that every other Serial.read() results in a -1 null character instead of reading the next character.

No, Serial.read returns -1 when there is nothing there to read, i.e. nothing has been received. (-1 is most definitely NOT null)

Follow your code through:
In loop, suddenly "available" returns 1 because a character has just been received.
Call ReadData, set c to zero, test to see if c is a semicolon (it isn't), so read the character that just came in.
it isn't -1, so print it.
Go back to the top of the while, see if the character was a semicolon.
It wasn't a semicolon, so read the the next character.
Oh dear. It hasn't arrived yet.

AWOL:

But I don't, I get this instead,
49
end
50
end
51
end

That's precisely whaty I'd expect you o get.
Serial comms are SLOW, you can whip around your while loop loads of times in the time it takes to receive one character (in fact, it's only your serial prints that are slowing you down)

See, I thought that from the very beginning, So I removed the serial printing and made it only print when nothing else was going on just to verify the data. all I got back was 0's as if nothing had been stored. I could only get the data to store if I printed it right away. So I need to verify that there is actually a character present before reading that character.