Increased SERIAL_BUFFER_SIZE to no effect

Hi

I'm reading in a NMEA string from a serial port. Here is my code:

void get_nmea_feed()
{
	//Setup. Serial Index is for keeping track of array position,
	serial_index=0;
	cr=false;
	memset(response, 0, sizeof(response) / sizeof(response[0]));		//clear input array
	//delay(600);			//This delay is for the module to finish.
	
	

	while (!cr)		   //while CARRIAGE RETURN is FALSE
	{
		incoming=nmea_port.read() ; //GET CHAR from buffer
		switch (incoming)
		{
			case 	'\r':
			cr=true; //CR= serial is done.
			//serial_index=0;
			break;
			
			case  '\n':
			break; //newlines are not needed in the buffer.
			
			case '?':
			break;
			
			default:
			if(serial_index<sizeof(response)-1)
			{
				response[serial_index]=incoming;
				serial_index++; 	   //STORE in response. Its a normal char.	
			}
				
		}
	}
	
	if(debug_pc){
		terminal.print("nmea feed: ");
		terminal.println(response);
		terminal.println(serial_index);
	}
}

It is adapted from another function which is used to receive responses from AT commands I send to the device . When I use it, I notice that the last few fields are cut off such that only 63 characters are actually put into the char array response. I searched here and then edited my HardwareSerial.h to have a SERIAL_BUFFER_SIZE 128 and it appears to have made no difference, its only accepting 63 characters. Some more details: I'm using Atmel Studio with visual micro on a DIGIX and the full string prints fine when I use

if(nmea_port.available())
	{
		terminal.write(nmea_port.read());
	}

Why would this be? Is my code bad or do I need to change the serial buffer some other way? Any help is much appreciated.

Added some info:
nmea_port is hardware serial.

I should add that what is happening isnt that nothing is being stored in my character array after index 63 , its that after character 63 it becomes "$G????????????????????????..."etc.
The NMEA format I'm using is GPGGA. (http://www.hemispheregps.com/gpsreference/GPGGA.htm). I've tried adding delays or changing the response variable size, makes no difference.

We need all your code. Are you using Serial or Software Serial for your reading?

You should be able to just create your own buffer, read characters until you see a '$' and then read from the device into your buffer. No need to have the serial port buffer everything.

All (lower case) string in c are terminated by a null char '\0' . Your not doing this so println if goofing up. you also need to take car of the linefeed char.

Mark

nmea_port looks like a SoftwareSerial instance. If it wasn't, it would be named Serial, Serial1, etc. Changing the size of the hardware serial buffer is going to have no impact on the SoftwareSerial buffer size. And, in general, is the wrong answer. Reading more frequently is.

KeithRB:
We need all your code. Are you using Serial or Software Serial for your reading?

You should be able to just create your own buffer, read characters until you see a '$' and then read from the device into your buffer. No need to have the serial port buffer everything.

All the code would be way too long to paste. I am using Serial.

holmes4:
All (lower case) string in c are terminated by a null char '\0' . Your not doing this so println if goofing up. you also need to take car of the linefeed char.

Mark

In the device schematics it specifically says the NMEA sentence is terminated with a (which is why I have the switch statement). The NMEA port is constantly printing (every second), it never stops. I'm unsure what println you are referring to, the only println in this function is the debug messages.

PaulS:
nmea_port looks like a SoftwareSerial instance. If it wasn't, it would be named Serial, Serial1, etc. Changing the size of the hardware serial buffer is going to have no impact on the SoftwareSerial buffer size. And, in general, is the wrong answer. Reading more frequently is.

I am using hardware serial.

#define terminal Serial
#define nmea_port Serial2
#define command_port Serial3

sean_8:

KeithRB:
We need all your code. Are you using Serial or Software Serial for your reading?

All the code would be way too long to paste. I am using Serial.

Then attach it to your post, or make a minimal sketch that shows the failure.

holmes4:
All (lower case) string in c are terminated by a null char '\0' . Your not doing this so println if goofing up. you also need to take car of the linefeed char.

In the device schematics it specifically says the NMEA sentence is terminated with a (which is why I have the switch statement). The NMEA port is constantly printing (every second), it never stops. I'm unsure what println you are referring to, the only println in this function is the debug messages.

Unless the incoming data is terminated with a NULL, it doesn't matter.
ALL C style strings MUST be terminated with a NULL (0) byte.
If you have an unterminated string, you can be guaranteed that something like terminal.println(response); will print everything up to and including the last thing you put into response[], and continue along its merry way, cheerfully spitting out garbage until it finds a NULL. Terminate your strings!

Is terminal.println(response); a debug message? It happens to be the response from the GPS, and that's what you are seeing that is crapping out.

I am using hardware serial.

#define terminal Serial

#define nmea_port Serial2
#define command_port Serial3

It is just a debug message, I use the char array "response" later on. I see what you're saying, for some reason I thought that the C++ compiler automatically added the terminator at the end for me implictly. eg char otherArray[10]={'a','r','d','u','i','n','o'}; but perhaps this only happens when the array is initialized rather than written to from Serial. Even so, if the null terminator would have occurred after the CR or LF that ended the sentence (where the switch statement was ending as it received CR or LF, so the null terminator wouldn't get put into the array), I still wouldn't have been able to place it into "response" because I never ended up getting it to begin with.

However even with that mind, where is the rest of the string? Shouldnt the whole string be printed, and then the garbage rather than just 64 characters? A nmea sentence like this should be ~100 characters long.

I tried a few things out and found that it worked with "readStringUntil('\n'). I guess I could have used "strcpy" to make the returned string into a char array but instead I looked at the source for readStringUntil (I hope that is allowed) and then added it to my program (timedRead was listed as private) so I added that too.

So this works:

void get_nmea_feed()
{
	//Setup. Serial Index is for keeping track of array position,
	serial_index=0;
	memset(response, 0, sizeof(response) / sizeof(response[0]));		//clear input array	
	//Get First Character	
	inc = timedRead(); 
	while(inc>=0 && inc!='\n')
	{
		if(serial_index<sizeof(response)-1)		 //if index is less than array max
		{
			response[serial_index]=inc;
			serial_index++; 	   //STORE in response. I
		}
		inc=timedRead();
	}
	if(debug_pc){
		terminal.print("nmea feed: ");
		terminal.println(response);
		terminal.println(serial_index);
	}
}
int timedRead()
{
	
	long start = millis();
	do {
		potential_incoming = nmea_port.read();
		if (potential_incoming >= 0) return potential_incoming;
	} while(millis() - start < 1000);
	return -1;     // -1 indicates timeout
}
	while(inc>=0 && inc!='\n')

NMEA sentences generally end with a *, not a carriage return.

Well, I'm still flying blind here, because I still don't see your entire code. What I do see, though, is that unterminated strings were the main cause of your problems. You ask "Where was the rest of my string? Well, have a look at the two sizeof() functions in the first code you posted. The sizeof() function will scan the string, looking for a NULL, and if there isn't one it will get the wrong answer.

BTW, my $GPGGA sentences are 81 characters long.

When I grab NMEA data, I look for a '$', then a '\r'. These characters do not need to be in the string, so I don't put them in there. I use the '$' to set a bool indicating I am collecting data, and to set my index to 0, and the '\n' to set a variable telling the rest of my program that I have a complete sentence.

Depending on what I am doing, while I am collecting the sentence, I will either NULL terminate the string after incrementing the index, or NULL terminate it when I get the '\r'. It's easy enough to just use:

        response[serial_index]=incoming;
        serial_index++; 	   //STORE in response. Its a normal char.	
        response[serial_index] = '\0';

You are wondering about the NULL termination, and when it happens. You can think of it this way.

An array of char is just an array.
A string (not to be confused with a String) is an array of char with a terminating NULL.

Initializing a char array with individual characters will not NULL terminate what you are thinking is a string. When you initialize with a string constant, like "I'm NULL terminated", the srray of char will be NULL terminated, since a string constant is NULL terminated.

If you put characters into an array, they do not generate a NULL right after them, and it's a good thing, too, because you are dealing with characters in that case. Various functions or marcos will NULL terminate the string, strcpy(), strcat(), itoa(), to name just a few.

I tried a few things out and found that it worked with "readStringUntil('\n'). I guess I could have used "strcpy" to make the returned string into a char array but instead I looked at the source for readStringUntil (I hope that is allowed) and then added it to my program (timedRead was listed as private) so I added that too.

You have to delete that code right away! It's so private you shouldn't even read it!

Just messin' with ya. 8)
Private means that you can't access that function from outside the library. It will be accessed by other functions within the library. What's allowed is basically whatever works.

Now, the reason we'd like to see your entire code... We don't know how you've declared variables, and we don't know what the rest of your code is doing, and how often you call get_nmea_feed(). Since your function is a blocking one, the rest of your code can't run while you are collecting a sentence. Depending on when, exactly, you call the function, you could well be right in the middle of a sentence in the serial buffer.

I would gather my data in loop(). You can still make a separate function you call to gether the sentence, but you should not make it a blocking function. Try something like this (untested):

void loop() {
  get_NMEA()
}

byte indx;
bool getting = false;
bool got = false;

void get_NMEA() {
  if gps_serial.available {
    char c = gps_serial.read();
    if (c == '

Note that you call it every time through loop(). In loop() you should test got and deal with the sentence. When you've finished with that data, set got = false, and your next sentence will be gathered.

PaulS, they actually end with a CR, which comes right after the checksum, which is right after the ''. But you can, of course, ignore the checksum and use the '' to indicate the end.) {
     getting = true;
     return;
   }
   if (getting && !got) {
     if (c == '\r') {
       got = true;
       getting = false;
       return;
     }
     if (c == 'n' || c == '?') {
       return;
     }
     response[indx] = c;
     indx++;
     response[indx] = '\0';
   }
 }
}


Note that you call it every time through loop(). In loop() you should test **got** and deal with the sentence. When you've finished with that data, set **got = false**, and your next sentence will be gathered.

**PaulS**, they actually end with a CR, which comes right after the checksum, which is right after the '*'. But you can, of course, ignore the checksum and use the '*' to indicate the end.

lar3ry:
The sizeof() function will scan the string, looking for a NULL, and if there isn't one it will get the wrong answer.

Not in any C language I've ever used.
sizeof() is a language operator that works at compile time.

I think you confusing this with the strlen() function.

--- bill

bperrybap:
I think you confusing this with the strlen() function.

Indeed I was thinking of strlen().
Thanks.