Memory issue?

Hello,

as an exercise I've written a sketch to take ascii data in on the serial port on an Arduino Uno and convert it to Morse code. I'm finding that if I comment out some of the code in a routine producing help text it works, if not commented out, it does not. The code is here:

http://petezilla.webdev.fire/public/embedded/arduino/morse

(too long to sensibly post in the forum).

The block of code I've commended out runs from lines 256 - 278.

I'm wondering if the micro-controller is running out of memory? I've produced the sketch with no particular regard for memory useage. However, the code resides in program memory, not SRAM, so I'm not certain it is a SRAM issue.

Thoughts?

Link is broken/incomplete

Are you in the default arduino environment programming? If so, you can see in the bottom of the screen how large your sketch is and how much memory available is.

Are you dynamically allocating memory with malloc functions?

If you have any large tables that aren't defined as being in PROGMEM, then they will use RAM.

To find out what RAM is being used

open a DOS window
navigate to the build folder, (to find this hold SHIFT while compiling your code). The path will be something like

C:\Users<yourname>\AppData\Local\Temp\build3014820664802798593.tmp

run

avr-size .cpp.elf

You’ll get a result like this

text data bss dec hex filename
1010 0 9 1019 3fb .cpp.elf

If data and bss add up to anything like maybe 1800 (hard to tell what, it depends on the program) you’re in trouble


Rob

Groove: Link is broken/incomplete

Sorry, I'm an idiot, that link was to the off-line (dev) version of my website. Correct link is:

http://www.petezilla.co.uk/public/embedded/arduino/morse

Specifically:

http://www.petezilla.co.uk/public/embedded/arduino/morse/morse7.pde

With the code commented out

text data bss dec hex filename 9656 760 201 10617 2979 blink.cpp.elf

with code included

text data bss dec hex filename 9860 1550 201 11611 2d5b blink.cpp.elf

That's 1751 bytes of RAM used not counting the stack. It's getting in the dodgy area.

Try just removing a few lines to see if that fixes the probem.


Rob

Groove:
If you have any large tables that aren’t defined as being in PROGMEM, then they will use RAM.

I have used a number of strings, using the string object. They are not especially large, but on the other hand there is not a lot of SRAM. The code I comment out to make things work is in the serial routines, with fixed strings.

Given that my variables are altered during the course of running the program it would not be a good idea to use PROGMEM I would have thought. I wonder however whether putting the help information below into a variable stored in program memory might help. Another solution, probably the best, would be to put it into the EEPROM. However, I’m not sure if there is a way that data could be included in the sketch. I assume I’d have to upload it separately.

However, I’m assuming it is a memory issue - am I right?

Routine below:

void help() {
  Serial.println("~");
  Serial.println("~USB ASCII to Morse converter.");
  Serial.println("~");
  Serial.println("~Help:");
  Serial.println("~ ##H: Print help (upper case H).");
  Serial.println("~ ##1: Rate 5 words per minute");
  Serial.println("~ ##2: Rate 10 words per minute");
  Serial.println("~ ##3: Rate 15 words per minute");
  Serial.println("~ ##4: Rate 20 words per minute");
  Serial.println("~ ##5: Rate 40 words per minute");
  
  
  /*
  If the following is not commented out then the arduino produces no
  output.
  */
  
  /*
  Serial.println("~");
  Serial.println("~Add # to the end of a line to repeat message.");
  Serial.println("~A # before any letter omits spaces to they can be run-");
  Serial.println("~together to generate prosigns, e.g. #CT (start of message).");
  Serial.println("~Carriage return (13) causes message to be transmitted.");
  Serial.println("~");
  Serial.println("~Commands and Messages:");
  Serial.println("~Lines beginning in tilde ~ are information");
  Serial.println("~Prompt for user input is >");
  Serial.println("~Messages:");
  Serial.println("~    <Ready    =    Awaiting commands or data.");
  Serial.println("~    <Sending  =    Sending morse.");
  Serial.println("~    <Wait     =    Please do not send data, buffers near full.");
  Serial.println("~    <Overrun  =    Too much information in buffers.  Contents ");
  Serial.println("~                   deleted without being sent.  Send data again");
  Serial.println("~                   in smaller chunks.");
  Serial.println("~");
  Serial.println("~Basic idea is you send data, when a <Wait is received then");
  Serial.println("~continue passing current word then hit return to send chunk");
  Serial.println("~before sending more data.");
  Serial.println(""); 
  */
  
  
}

Graynomad:
With the code commented out

text data bss dec hex filename
9656 760 201 10617 2979 blink.cpp.elf

with code included

text data bss dec hex filename
9860 1550 201 11611 2d5b blink.cpp.elf

That’s 1751 bytes of RAM used not counting the stack. It’s getting in the dodgy area.

Try just removing a few lines to see if that fixes the probem.


Rob

Interesting, with the help commented out as below I can send 10 characters but with 20 it hangs. I suspect that this is it running out of memory - perhaps a buffer over-run type problem, not that I have a fixed length buffer, but I do have finite ram.

void help() {
  Serial.println("~");
  Serial.println("~USB ASCII to Morse converter.");
  Serial.println("~");
  Serial.println("~Help:");
  Serial.println("~ ##H: Print help (upper case H).");
  Serial.println("~ ##1: Rate 5 words per minute");
  Serial.println("~ ##2: Rate 10 words per minute");
  Serial.println("~ ##3: Rate 15 words per minute");
  Serial.println("~ ##4: Rate 20 words per minute");
  Serial.println("~ ##5: Rate 40 words per minute");
  
  
  /*
  If the following is not commented out then the arduino produces no
  output.
  */
  
  
  Serial.println("~");
  Serial.println("~Add # to the end of a line to repeat message.");
  Serial.println("~A # before any letter omits spaces to they can be run-");
  Serial.println("~together to generate prosigns, e.g. #CT (start of message).");
  Serial.println("~Carriage return (13) causes message to be transmitted.");
  Serial.println("~");
  Serial.println("~Commands and Messages:");
  Serial.println("~Lines beginning in tilde ~ are information");
  Serial.println("~Prompt for user input is >");
  
  /* 
  
  Serial.println("~Messages:");
  Serial.println("~    <Ready    =    Awaiting commands or data.");
  Serial.println("~    <Sending  =    Sending morse.");
  Serial.println("~    <Wait     =    Please do not send data, buffers near full.");
  Serial.println("~    <Overrun  =    Too much information in buffers.  Contents ");
  Serial.println("~                   deleted without being sent.  Send data again");
  Serial.println("~                   in smaller chunks.");
  
  
  
  Serial.println("~");
  Serial.println("~Basic idea is you send data, when a <Wait is received then");
  Serial.println("~continue passing current word then hit return to send chunk");
  Serial.println("~before sending more data.");
  Serial.println(""); 
  */
  
  
}

koenwar: If so, you can see in the bottom of the screen how large your sketch is and how much memory available is.

This is not the case. This message is the amount of Flash (or program) storage being used and what is left. It has nothing to do with the RAM at Runtime.

It is entirely possible to have a sketch that takes up 15k of Flash/Program Memory and only use a couple hundred bytes of RAM.

All those Serial.print strings are wasting RAM - move them to PROGMEM.

AWOL: All those Serial.print strings are wasting RAM - move them to PROGMEM.

You mean something like the string demo in:

http://www.arduino.cc/en/Reference/PROGMEM

From what you are suggesting then, if I have a line:

Serial.println("Hello world!");

then the complied program puts "Hello world!" in RAM, uses it and does not release it? Just curious.

then the complied program puts "Hello world!" in RAM, uses it and does not release it?

How could it "release it"?

AWOL:

then the complied program puts "Hello world!" in RAM, uses it and does not release it?

How could it "release it"?

Perhaps a bad choice of words. However, the implication that my many Serial.println command consume RAM implies that those RAM locations are not reused. I.e.

Say "Hello World" is written to memory locations say 100-110 (decimal). These are now output to serial. Now I want to send to the serial port "Hello world again" that implies that instead of writing from program memory to RAM "Hello World Again" in memory locations say 100-116 that it is writing them in locations 111-127. 100-110 is not being re-used. Seems odd to me.

Pete

It would stand to reason that the complier would only store one copy of a constant, but I do not know if this is the case. In your code each of your string constants are different.

The complier does not know how many times a constant (string or otherwise) will be used (what if there was a loop?), so the constant has to be stored in RAM at all times.

By moving the (string) constants into PROGMEM, you never actually copy the constant into RAM.

Its a series of Serial.println("") statements, see below. So I’m not writing to a series of constants at all. It must be the way the compiler does things.

  Serial.println("~");
  Serial.println("~USB ASCII to Morse converter.");
  Serial.println("~");
  Serial.println("~Help:");
  Serial.println("~ ##H: Print help (upper case H).");
  Serial.println("~ ##1: Rate 5 words per minute");
  Serial.println("~ ##2: Rate 10 words per minute");
  Serial.println("~ ##3: Rate 15 words per minute");
  Serial.println("~ ##4: Rate 20 words per minute");
  Serial.println("~ ##5: Rate 40 words per minute");

AWOL:
All those Serial.print strings are wasting RAM - move them to PROGMEM.

Re-written help, whole sketch was using 1400 bytes without part commenting out. I’m not sure if I’ve done this correctly.

void help() {
  
  prog_char string_0 [] PROGMEM = "~USB ASCII to Morse converter.~";
  Serial.println(string_0);
  
  prog_char string_1[] PROGMEM = "~Help:";
  Serial.println(string_1);
  
  prog_char string_2[] PROGMEM = "~ ##H: Print help (upper case H).";
  Serial.println(string_2);
  
  prog_char string_3[] PROGMEM = "~ ##1: Rate 5 words per minute";
  Serial.println(string_3);
  
  prog_char string_4[] PROGMEM = "~ ##2: Rate 10 words per minute";
  Serial.println(string_4);
  
  prog_char string_5[] PROGMEM = "~ ##3: Rate 15 words per minute";
  Serial.println(string_5);
  
  prog_char string_6[] PROGMEM = "~ ##4: Rate 20 words per minute";
  Serial.println(string_6);
  
  prog_char string_7[] PROGMEM = "~ ##5: Rate 40 words per minute\n~";
  Serial.println(string_7);
  
  prog_char string_10[] PROGMEM = "~Add # to the end of a line to repeat message.";
  Serial.println(string_10);
  
  prog_char string_11[] PROGMEM = "~A # before any letter omits spaces to they can be run-";
  Serial.println(string_11);
  
  prog_char string_12[] PROGMEM = "~together to generate prosigns, e.g. #CT (start of message).";
  Serial.println(string_12);
 
  /* 
  prog_char string_13[] PROGMEM = "~Carriage return (13) causes message to be transmitted.\n~";
  Serial.println(string_13);
  
  prog_char string_20[] PROGMEM = "~Commands and Messages:\n~";
  Serial.println(string_20);
  
  prog_char string_21[] PROGMEM = "~Lines beginning in tilde ~ are information";
  Serial.println(string_21);
  
  prog_char string_22[] PROGMEM = "~    <Ready    =    Awaiting commands or data.";
  Serial.println(string_22);
  
  prog_char string_23[] PROGMEM = "~Prompt for user input is >";
  Serial.println(string_23);
  
  prog_char string_24[] PROGMEM = "~    <Sending  =    Sending morse.";
  Serial.println(string_24);
    
  prog_char string_25[] PROGMEM = "~    <Wait     =    Please do not send data, buffers near full.";
  Serial.println(string_25);
  
  prog_char string_26[] PROGMEM = "~    <Overrun  =    Too much information in buffers.  Contents ";
  Serial.println(string_26);
  
    
  prog_char string_27[] PROGMEM = "~                   deleted without being sent.  Send data again";
  Serial.println(string_27);
  
  prog_char string_28[] PROGMEM = "~                   in smaller chunks.";
  Serial.println(string_28);
  */
    
}

I'm not particularly impressed by the PROGMEM examples that still want to use strcpy_P() to copy to a RAM buffer before then using the string. This STILL requires unnecessary buffers. For my part, I have done this sort of thing:

void print_P(const char *s)
{
  uint8_t c;
  while ((c = pgm_read_byte_near(s++)) != 0)
     Serial.print(c);
}

static const PROGMEM char long_msg[] =
  "message line one\r\n"
  "message line two\r\n"
  "message line three\r\n"
  "message line four\r\n"
  "message line five\r\n"
  "message line six\r\n";
  print_P(long_msg);

It's a shame that the PROGMEM attribute is not "visible" at the C++ method signature level, so that you could simply implement methods to handle it. It would be nice to have some extensions to String and print that would allow static PROGMEM strings to be used more conveniently. Something to work on, I guess...

gardner: I'm not particularly impressed by the PROGMEM examples that still want to use strcpy_P() to copy to a RAM buffer before then using the string. This STILL requires unnecessary buffers. For my part, I have done this sort of thing:

Thanks Gardner, I'll have to look into that. I'm not exactly a C/C++ guru so I don't get all the details of the languages and ardunio enironment yet.

Pete

It would stand to reason that the complier would only store one copy of a constant, but I do not know if this is the case.

I think it is if they are the same.

I just changed a table of strings from (bear in mind there is about 50 of these)

{"7", "6", "5", "4", "3", "2", "1", "0"}

to

{"BIT7", "BIT6", "BIT5", "BIT4", "BIT3", "BIT2", "BIT1", "BIT0"}

And it made almost no difference to the space used by the array. The size when up by 24 bytes IE sizeof("BIT") x 8.

So only a single copy of each string was being stored, despite them being used 50 times.

Back in the day of 2k program memory and all assembly code I used to tokenise common words, for example a string might look like this

0x1, "value entered is", 0x2, "for", 0x1, 0x2, 0x3, 0x0

the non-ASCII values in a string were indexes into an array of common words. eg

{"The", "not valid", "Bushtronics", "system"}

the print would detect the non-ASCII values and pull strings from the array

"The value entered is not valid for the Bushtronics system"

In this example I've saved 27 bytes.

Fortunately I don't think we have to be this extreme these days.


Rob