I've already received some help on the forum regarding this topic for which I thank those concerned.
I'm in the process of constructing a project using an Arduino Mega equipped with GSM and Data-Logging Shields, the latter has an SD Card Reader and an RTC.
I collect NMEA 'sentences' from two sources and save the data two text files, one contains GPS and the other Water Depth. Each sentence can be upto 80 characters in length including delimiters. I add a timestamp at the beginning of each sentence thus yymmddhhmmss therefore maximum total length might be 92 characters although some of the data fields in the sentences might not contain data. Each file then could be upto 8GB in size.
At midnight each day the current files are closed and new ones opened for the following day; SMS messages are sent during this midnight routine to alert me of success/failure of the process. The kit will be deployed in a remote location and not easily accessible.
My final task now is to transfer the data to a remote computer for processing. I have discovered that FTP is a non-starter so I think my only option is via HTTP POST and the guy responsible for the remote computer prefers that method.
Because the data files are quite large transfer is going to take a long time and I'd like to ask for some advice.
I think I have a number of options:
- Read an NMEA sentence from the data file and then send it a character at a time to the remote end. Do this for each data file, sentence by sentence
- Read the entire datafile character by character and send it as one endless stream.
- Forget the data files and simply transfer the incoming data streams 'on the fly' directly to the remote end. This might cause problems because the data streams would be converged into one, i.e. GPS-DEPTH-GP-DEPTH etc. I am unsure if it is possible to open two POST connections and send the GPS and DEPTH data in two unique streams.
Then there is the timing to consider. Each NMEA unit sends its data as a 'sentence' once a second for each stream.
I have a lot to learn about the POST method of transfer so am on a steep learning curve.
Am I expecting too much from this setup? Is the GSM module capable of transferring so much data efficiently?
I thanks you for any advice or criticism that will help me.