Reading NMEA data from a serial port.

Hi,
Is it possible to read data from an input port without ‘destroying’ the data at the same time?

Let me explain what I want to do, I’d like to read in NMEA sentences from a serial port, validate the sentence from the last two check digits, and then if the data is valid re-transmit it over a Bluetooth link.

Because I want to check that it’s valid before I re-transmit, it seems I have to read the data in as it arrives, store it in a buffer as long as the longest sentence I may receive, and then transmit the data after it’s been validated. This requires allocating a lot of memory to store the characters of the sentence as they arrive.

I know that incoming data is stored internally in a 128 character buffer, and as soon as the data is read that character is removed from the buffer. What I’d like to do (if possible) is to read the data in the buffer without removing it from the buffer. I’d like to wait till a whole sentence is in the buffer, then check the last two checksum characters, and then (if valid) read and re-transmit the whole sentence again one character at a time, without the need to provide storage for the whole sentence.

Is this possible? Or am I just going to have to bite the bullet and allocate another 128 bytes just for storing each sentence?

Thanks

I know that incoming data is stored internally in a 128 character buffer

That's a dangerous, IDE-dependent assumption

This requires allocating a lot of memory to store the characters of the sentence as they arrive.

Did you need Memory Efficient solution,why?

That's a dangerous, IDE-dependent assumption

I'm working with a Arduino Mega. I believe that 128 characters is the correct value.

Did you need Memory Efficient solution,why?

Most of my projects seem to end up running low on memory, so I try to be memory efficient from the start. This particular project also involves sending and receiving SMSs which also involve storing long strings so I've got an extra need for prudence.

thanks

I'm working with a Arduino Mega. I believe that 128 characters is the correct value.

Why "believe", when you can know? - you have the source.
The serial buffer size (SERIAL_BUFFER_SIZE) depends on the IDE, not the processor (except for the very small processors), and for IDEs after 1.0 is 64 bytes.

Most of my projects seem to end up running low on memory, so I try to be memory efficient from the start. This particular project also involves sending and receiving SMSs which also involve storing long strings so I've got an extra need for prudence.

124K Flash
8 KB SRAM

Use FLash for "sending and receiving SMSs" Like

FLASH_STRING(big_string, TASK 1, TASK 2, TASK 3...)

I think this will solve your problem of Extra usage "memory for 128 characters". 8 KB SRAM enough for project like that if you use intelligently memory. 8)

cheers

serial buffer size (SERIAL_BUFFER_SIZE) depends on the IDE, not the processor

How can this be the case? The serial buffers on each of the input ports is a hardware thing isn't it?????

Use FLash for "sending and receiving SMSs"

I had understood that flash memory (unlike SRAM) can only be written to a finite number of times?
My SMS application also stores up to 20 unsent SMS when in poor reception areas, which bumps up memory usage quite dramatically. (I know I could write to an SD card, but that adds another layer of complexity.)

Cheers

The serial buffers on each of the input ports is a hardware thing isn't it?????

No.

Have you had a look at the TinyGPS library? Its quite efficient and written by a pro so may be more efficient than one you write yourself. If you don't want all its features then just extract the parsing component of the library.

http://arduiniana.org/libraries/tinygps/

Have you had a look at TinyGPS?

Yes - it reads each sentence into an 80 character buffer before processing, which was what I was hoping to avoid!

The serial buffers on each of the input ports is a hardware thing isn't it?????

No.

Isn't the serial port buffer a hardware buffer built into the chips UART? If not, I've obviously got the wrong idea of what's going on here, where can I read up on this?

Thanks

Isn't the serial port buffer a hardware buffer built into the chips UART?

No.

If not, I've obviously got the wrong idea of what's going on here, where can I read up on this?

Section 19 of the AVR datasheet.
I don't think I've ever come across a small-device USART that had more than about 8 bytes of FIFO.

My SMS application also stores up to 20 unsent SMS when in poor reception areas, which bumps up memory usage quite dramatically.

Long ago we use "DGS 100 3G Module" unsent SMS stored in SIM and by viewing the status of message you will re-send them OR your message center will attempt to re-send the message at a later time. 8)

80 character buffer before processing, which was what I was hoping to avoid!

Are you that strapped for RAM?

You could reduce the buffer size to say 4 (assuming HardwareSerial will allow that) and add your 80-byte array, net result only 20 extra bytes of RAM.


Rob

am I just going to have to bite the bullet and allocate another 128 bytes just for storing each sentence?

This, pretty much, or at least as much as is required for the longest NMEA sentence. Since you're using a mega, that's not a lot, and while it's obviously good practice to be frugal with memory, it seems like you're worrying too early. Don't forget to use the F macro where appropriate to put constant strings in progmem.

How about storing the sentences on an external eeprom chip? Plenty of room for your requirements without clogging up your AtMega chip.

http://www.arduino.cc/playground/code/I2CEEPROM