Sending NMEA Data using GSM Shield

Projects Networking, Protocols, and Devices
1

I've already received some help on the forum regarding this topic for which I thank those concerned.

I'm in the process of constructing a project using an Arduino Mega equipped with GSM and Data-Logging Shields, the latter has an SD Card Reader and an RTC.

I collect NMEA 'sentences' from two sources and save the data two text files, one contains GPS and the other Water Depth. Each sentence can be upto 80 characters in length including delimiters. I add a timestamp at the beginning of each sentence thus yymmddhhmmss therefore maximum total length might be 92 characters although some of the data fields in the sentences might not contain data. Each file then could be upto 8GB in size.

At midnight each day the current files are closed and new ones opened for the following day; SMS messages are sent during this midnight routine to alert me of success/failure of the process. The kit will be deployed in a remote location and not easily accessible.

My final task now is to transfer the data to a remote computer for processing. I have discovered that FTP is a non-starter so I think my only option is via HTTP POST and the guy responsible for the remote computer prefers that method.

Because the data files are quite large transfer is going to take a long time and I'd like to ask for some advice.

I think I have a number of options:

  • Read an NMEA sentence from the data file and then send it a character at a time to the remote end. Do this for each data file, sentence by sentence
  • Read the entire datafile character by character and send it as one endless stream.
  • Forget the data files and simply transfer the incoming data streams 'on the fly' directly to the remote end. This might cause problems because the data streams would be converged into one, i.e. GPS-DEPTH-GP-DEPTH etc. I am unsure if it is possible to open two POST connections and send the GPS and DEPTH data in two unique streams.

Then there is the timing to consider. Each NMEA unit sends its data as a 'sentence' once a second for each stream.

I have a lot to learn about the POST method of transfer so am on a steep learning curve.

Am I expecting too much from this setup? Is the GSM module capable of transferring so much data efficiently?

I thanks you for any advice or criticism that will help me.

Kind regards.

2

Do you have to send the raw NMEA data? The parsed forms could be much smaller, in one record per second, or even binary or encoded binary.

Cheers,
/dev

3

Then there is the timing to consider. Each NMEA unit sends its data as a 'sentence' once a second for each stream.

Timing will be a problem. You spend all day collecting data, and expect to send it out in a reasonable time frame. I don't think that is realistic.

Am I expecting too much from this setup?

Probably.

Is the GSM module capable of transferring so much data efficiently?

No. It can deliver the data, but it is going to take a long time - hours probably.

Sending the data ll at once will not result in less data, nor will it take less time, or use less power, than sending the data as it is collected.

4

Thanks Guys, Seems like I have a lot to do!

I think I'll cut my sampling rate down to one a minute or even one every five mins until I've mastered the transfer.

Are you able to point me to some info on using the POST utility - I have no idea where to start and most of the examples/messages I've found seem to be using an ethernet shield whereas I'm using GSM

If I have something to start with I'll be able to understand the system better.

Thanks again.