Send long char array (7000) via hardware serial

Projects Programming
1

I'm trying to send POST data via a SIM900 module. The process is to create the large POST string by strcat'ing variables together, then set the SIM900 module up to send, then using hardware serial to output the char array @ 19200.

I'm using a ATMega1284 chip that has substantially more SRAM than a standard Arduino Uno, and I'm using the HobbyTronics Uno*Pro bootloader and pin layout. When I compile, I get this:

Sketch uses 30,432 bytes (23%) of program storage space. Maximum is 130,048 bytes.
Global variables use 12,774 bytes (77%) of dynamic memory, leaving 3,610 bytes for local variables. Maximum is 16,384 bytes.
Low memory available, stability problems may occur

It works for the first few hundred chars, but then corrupts. I've got to the point where I create the string, then I print it out 3 times with Serial.println() and each instance is different.

As I can't put it all in here, I've put the output and the code in this pastebin: ========OUTPUT========Print 1e[0]=453768097,57,0,18,0,0,0,12.05,1,0,29.3 - Pastebin.com

Any pointers on whats going on? I've tried everything I know, and I've hit a wall :frowning: I've increased the HardwareSerial buffer size, but that didn't seem to help either.

I'm using MsTimer2 with an interval of 150ms if that would affect the Serial interrupts.

2

I'm using MsTimer2 with an interval of 150ms

For what?

3

This setup is for a rather elaborate bike system, which needs to always be flashing its LED's. The GPRS part sends back its location and battery level.

Will that cause a problem?

4

Will that cause a problem?

That depends how you have coded it.

5

This setup is for a rather elaborate bike system, which needs to always be flashing its LED's.

Clearly some trailer queen that doesn't deserve to be on the road. So, whether any of it works, or not, doesn't really matter.

6

Ok, with a fresh head I started working on this again, and I started with MsTimer2. If I completely remove the library, my 3 Serial.println's all match.

This is odd, as I was using the ::stop, and ::start functions of MsTimer2 so I guess the actual setup of the system.

Whats a trailer queen? This is for a push bike, I just want the rear red LED's to flash at around 3Hz, which is well within any UK laws regarding additional bike lights.

7

Whats a trailer queen?

A bike that is destined to spend it's life on a trailer, being towed from place to place to be seen (not ridden).

This is for a push bike

A what?

I just want the rear red LED's to flash at around 3Hz, which is well within any UK laws regarding additional bike lights.

I don't see the need for a timer to manage this. I don't see ANY relationship between flashing the read LEDs on a bike and sending huge amounts of data via the serial port.

Just because a riding mower can be equipped with a blender to make margaritas doesn't mean that it should.

8

A bicyle: Bicycle - Wikipedia

As the processor is doing other things while travelling - collecting GPS data and collecting wheel rotations - I thought that the best way to always have a rock solid flash would be to use a timer. I tried to put it in a standard loop using essentially the same code as http://arduino.cc/en/Tutorial/BlinkWithoutDelay but other processing in the loop caused the frequency to change so that I could notice. I might have another look at doing it that way.

I sent a lot of data as I like to graph and plot where the bike has been, what the environment is etc. its a hobby, and its slowly gotten more and more complicated. The 7000 char length listed in the title is for the worst case scenario, I fill up a struct with events, then every hour I sent it over the SIM900 GPRS connection to save battery. Now that I've made some changes, I think the worst case scenario would be ~3000 chars if the SIM900 couldn't connect for 3 consecutive hours.

Just tested again, and the strings don't match; parts repeat and the length isn't quite the same.

Is it possible to get rid of this, or is it a fundamental limitation of the Arduino?

9

Is it possible to get rid of this, or is it a fundamental limitation of the Arduino?

It's more likely that it's a fundamental limitation of your code. Which you may have noticed that you haven't posted.

10

Attached.

The AsyncGPRS library is something I wrote myself while looking at a GPRSbee library by Kees Bakker. Just giving credit for the original code as I didn't expect I would be posted it online at the time.

GPRS_BikeV5Pro.zip (15.2 KB)

11

TenderLoins:
I'm trying to send POST data via a SIM900 module. The process is to create the large POST string by strcat'ing variables together, then set the SIM900 module up to send, then using hardware serial to output the char array @ 19200.

Whatever goes through serial shouldn't matter if it's all at once or in pieces, does it? The SIM900 gets serial data 1 char at a time... would a small delay between chars screw it up?

So instead of assembling a 7000 byte string and then sending it, why not just send the pieces that you would have been appending to the buffer? And for debug, use a terminal program on your PC to capture serial and maybe send up debug lines too.

Think of serial write as a kind of buffer you can feed bytes to.

12

This is for a push bike

Standard British English for a bicycle.

13 
typedef struct {
      unsigned long timestamp; // Timestamp when the event was generated
      int reason; // The reason why the event was added
      int state; // The LED state, wake, sleep etc.
      unsigned long rotations; // Number of rotations the wheel has done
      unsigned int on_time; // Number of seconds the device has been awake
      unsigned int half_on_time; // Number of seconds the lights have been half on
      unsigned int full_on_time; // Number of seconds the lights have been fully on
      float voltage; // The battery voltage
      unsigned int messages_sent; // The number of messages the device has sent
      unsigned int messages_failed; // The number of messages that were attempted but failed
      float temperature; // The temperature
      int moving; // Whether we're moving
      unsigned int event_number; // The event number
      
      double lat; // Current latitude
      double lng; // Current longitude
      unsigned int sats; // Number of satelites involved
      unsigned int age; // Milliseconds since last fix
} Events;

This looks like data for ONE event, not for multiple events Why does the struct have a plural name?

char post_data[7000];// = "1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&1234567890123456789012345678901234567890abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz&";
Lose the stupid comment.

  if (asyncbee.getStep() == 12) {
    asyncbee.addPOSTdata(post_data);

You haven't written anything to post_data. Passing an empty array to the method seems pointless.

14

You're right, the naming of my struct is not as clear as it should be. That comment was me testing out the function; it should have been removed. The asyncbee.addPOSTdata function does get data as the post_data array is filled prior to that function being called; the process is:

post_data is filled with Events
The asyncGPRS class is started and is constantly called and polled, and advances through multiple steps
Once step 12 is reached, the post_data is output via Hardware serial (which is connected to the SIM900 module)

However, even prior to that point, the post_data array does not output correctly when it is first created - lines 151 through 153 in events.ino.

why not just send the pieces that you would have been appending to the buffer

I could do that, and I probably will end up doing it, I just don't understand why I can't output a long string without corruption. Its really frustrating as this is the last thing I need to do before I can finish this project.

Thanks for the help so far everyone :slight_smile:

15

post_data is filled with Events

Where? I searched for post_data. I found the declaration and the call to addPOSTData().

16

If you have a look at line 142 in events.ino, the individual row is added to post_data. At least thats what I'm trying to do?

17

TenderLoins:

why not just send the pieces that you would have been appending to the buffer

I could do that, and I probably will end up doing it, I just don't understand why I can't output a long string without corruption. Its really frustrating as this is the last thing I need to do before I can finish this project.

Thanks for the help so far everyone :slight_smile:

That could be for hardware or software reasons. I'm still not clear on how it is going wrong but maybe that makes two of us?

Just looking at your Events struct, it looks like you use int as the smallest data unit even for a 1 bit flag?
But then you turn it all into text and have plenty of RAM, unless the problem is due to reset on memory crash, chip RAM holds the stack as well as the heap and deeply nested calls really grow the stack.

Pushing prints straight down serial would save you 7000 bytes RAM plus the time it takes to assemble 7000 bytes of data. Sorry but buffering to RAM is not free even when you have the RAM to spare.

18

Yeah, I create the string, then try and print it 3 times (this is for testing to show the issue) and the string is printed differently each time.

I would love to do it your way, outputting the serial data directly without the string middleman, but the SIM900 module needs to know the amount of data its going to send before you enter the data; i.e

  • Send 900 b of data
  • Here's the 900 b

As the amount of data changes each time, I don't know how long the string is prior to entering the data. Thinking about it, I might be able to figure that out with some basic maths... In fact you might be onto something there....

Its just annoying that I can't use all of that delicious RAM to do this the 'easy' way :slight_smile:

19

Ahhh. New conditions that change the situation.
Well then see about getting the code running first.
I can't duplicate your hardware, btw, so I can't test or debug.

20

Yeah sorry, I keep remembering why I did things a certain way. This has been a long project with me doing bits of it in the evenings, and I haven't been making notes on certain design choices....

I was thinking that if I knew the absolute maximum size that the post data could be, I could say to the module, 'Hey, I'm gonna send 10,000b' then use the return value of Serial.print to determine how much has been written, then subtract that figure from 10000 and add null characters, or spaces until I reach 10000. Inefficient, but should work.

Or I could do what I'm doing at the moment and find the length, but when I come to send the data, I rerun the code, but instead of strcat'ing it into a big long string, I output it via serial.

If you don't send the correct number of bytes you're going to write to the SIM900 module, it just returns an error.

Oh, and my variable types are poorly chosen, I should just be using char or byte for those single b entries. I hadn't got to the stage of it working before I optimised everything. I think my assuming that large amount of RAM will fix everything has been foolish.