next topic »

[on:]

First, I'm _not_ familiar with the specifics of the AVR compiler,
only in using the Arduino IDE. I am using IDE ver 1.0 with a
Duemilanove 328 bootloader chip.

I am writing a large sketch for my robot tank, and am up to a sketch
size of "15600 bytes (of 30720 byte maximum)", and currently about
half done. Everything has been running fine until the problems listed
below. I found nothing about this in the help reference or trouble-
shooting guide.

I have a multi-level menuing system with a lot of char* strings
being sent via Serial.print(), eg

Serial.print("\nServo Control Test ... ");
Serial.print("\ncmds: (q)uit, (s)top, (f)wd, (b)wd, (l)eft, (r)ight ");
.... on and on ....

I have 2 problems. First, I found if I Serial.print() a string of
length more than about 34-36 bytes, then it writes garbage characters
at the end, so I assume there is a 32-byte buffer for this function,
but did not see this fact mentoned anywheres. ????

Secondly and more serious, after implementing probably 30-40 such
Serial.print() statements in the various menus, the program started
hanging in the middle of printing the 'first' [startup] menu. It would
run fine if I commented out most of the Serial.print() statements,
however.

So, I assume I have run out of char* string storage space in the
program, and I am wondering what the limit is on that. ????

I imagine static string storage is overwriting the stack or data
variable space, or something. Can someone tell me how to determine
the program memory allocation for statics, data, variables, stack, etc.
????

Is it possible to change this ????

I did track down the avr-size.exe utility buried deep in the Arduino
IDE directory, but it requires an "a.out" file, which the Arduino IDE
does not produce.

Any way to fix all of this? Note that, during Verify, the IDE does not
print any error messages whatsover, indicating I have exceeded any
storage space limits, etc.

[Reply #1 on:]

Lots of strings being printed means you are probably exceeding the amount of SRAM (done it myself). The first thing to try, is to put those strings into program memory space (PROGMEM) and the easiest way to do that is to change each statement that looks like this:

Code:

Serial.print("whatever");

into this:

Code:

Serial.print(F("whatever"));

The F() macro puts the enclosed string in PROGMEM and Serial.print will be loaded with a function which prints the string from PROGMEM instead of SRAM.
It also works with Serial.println().

Pete

[Reply #2 on:]

Hi Pete, thanks for the response. I tried your suggestion briefly, but had some
compile time errors, and will try it some more tomorrow. I didn't mention it
last time, but my actual print functions are much more involved than just
Serial.print("......"), so I have to rewrite dozens of lines of code.

However, I don't understand your suggestion. It seems to me that most compilers
store strings such as "Servo Control Test ...", etc, somewheres in program memory
space [ie, Flash] and not in RAM. This is because they are static constant unchanging
data arrays, and in order to print them at all, they have to be stored permanently
for access at runtime. They may be moved into a smallish RAM buffer during printout,
but initially are not stored there, so ???

To further confuse me, I did track down the PROGMEM.html file. It is missing from
the reference directory of IDE 1.0, but is present in ver. 0022 directory. It mentions
a different scheme from yours for manually storing strings in PROGMEM space, but
as mentioned above, I don't even understand why it needs to be done, as the
compiler should place strings like "Servo Control ..." in Flash by default.

Well, tomorrow, more play.

[Reply #3 on:]

somewheres in program memory space [ie, Flash] and not in RAM. This is because they are static constant unchanging data arrays, and in order to print them at all, they have to be stored permanently
for access at runtime. They may be moved into a smallish RAM buffer during printout,
but initially are not stored there, so ???

Yes the strings are stored in flash.  However to be used they get copied into RAM.  The copying to RAM causes the poor little ATmega's heap to get fragmented very quickly.  It is best to keep strings in progmem and only copy them out one byte at a time.  This keeps SRAM from getting corrupted.  2K of RAM and lack of memory manager means being delicate with huge data structures.

I can't speak to the 32character limit. I have never used strings that long on an Arduino project.

As for your serial.print() functions being more complex than "...", I suggest simplifying them.  (though, I'm at a loss how you could make the function call "complicated")

[Reply #4 on:]

I would suggest OP move all the constant strings into PROGMEM.

I have an older blog post about optimizing programs:

http://liudr.wordpress.com/2011/02/04/how-to-optimize-your-arduino-memory-usage/

I'll add something with regard to the new F() directive.

[Reply #5 on:]

James C4S, thanks for the info. I guess one must expect some RAM fragmentation problems with
small RAM space available. I'll just play around till I find something that works ok. I should think
the following 'might' work. Sets of strings defined at the top of each menuing file, and read out
one character at a time inside the menus. Doesn't require any fancy F() macros, and avoids
having to use overly-complex PROGMEM constructs.

<pre>
#define MSSIZ  4
const char* MsgServo[MSSIZ] = {
  "Servo Control Program ...",
  "cmds: (s)top, (f)wd, (b)wd, (l)eft, (r)ight",
  ... on and on ...
};

void prnMsgServo()
{
  char ch;

  for( int i=0; i<MSSIZ; i++) {
    int j=0;
    while( (ch=MsgServo[j++]) != 0) cWrite(ch); //see below for cWrite().
  }
}
</pre>

Ok, I just tried the above and it works fine. I'll see if I can get my entire menuing system to
work ok with this scheme.

The reason my print functions are "more complicated" is because my robot tank has XBee to
make remote comms with the PC. Therefore I defined 2 sets of print functions that I can #ifdef
in between development [uses Serial.print() via USB] and remote tank operation [uses software
uart and U2.print() via XBee]. Eg,

<pre>
#ifdef U1_SINGLE
void sPrint(char* s)
{
  Serial.print(s);
}
#endif

#ifdef U2_SINGLE
void sPrint(char* s)
{
  U2.print(s);
}
#endif
</pre>

cWrite() and iPrint() are analogous for characters and integers.

Also, as I don't like long sets of print statements, I defined the following, with the above
functions embedded inside. Saves quite a lot of space.

<pre>
// usage: printf_sis(" [var1=", -123, "] ");
void printf_sis(char* s1, int num, char* s2)
{
  sPrint(s1);
  iPrint(num);
  sPrint(s2);
}
</pre>

There are others which are similar to printf_sis().

printf_si(...)
printf_sisi(...)
..etc..

And the F() macro doesn't seem to work from the top-level calls, only inside
Serial.print( F(.) ) directly.

I'll also try rewriting sPrint ...

<pre>
void sPrint(char* s)
{
  char ch;
  int i=0;

  while( (ch=s[i++]) != 0) cWrite(ch);
}
</pre>

Thanks for the help.

[Reply #6 on:]

liudr, thanks for the info, very helpful. Especially,

-------
“Hello” will be copied to the SRAM so taking space in both flash and SRAM.
This is unfortunate for variables like an integer but constant strings
don’t change their values so they don’t need to reside in SRAM at all time.
-------

Since you're talking about memory usage, possibly you could add something about using
avr_size.exe, using verbose compile mode, and explaining what the following (.text, .bss, etc)
all means - or is this out of bounds for what you're up to?

C:\>avr-size -C --mcu=atmega328p svt.elf
AVR Memory Usage
----------------
Device: atmega328p

Program:   15720 bytes (48.0% Full)
(.text + .data + .bootloader)

Data:       1564 bytes (76.4% Full)
(.data + .bss + .noinit)

Thanks for the help.

[Reply #7 on:]

oh sorry, I see this statement was wrong:

while( (ch=MsgServo[j++]) != 0) cWrite(ch); //see below for cWrite().

s/b

while( (ch=MsgServo[j++]) != 0) cWrite(ch); //see below for cWrite().

[Reply #8 on:]

liudr, thanks for the info, very helpful. Especially,

or is this out of bounds for what you're up to?

I'm afraid so but willing to learn

Is "svt.elf" a file name or a fancy option?

[Reply #9 on:]

liudr, thanks for the info, very helpful. Especially,

or is this out of bounds for what you're up to?

I'm afraid so but willing to learn

Is "svt.elf" a file name or a fancy option?

file name; if you enable verbose output while compiling (in the preferences menu) you can see where the file is. It will be the name of your sketch (without the .ino or .pde) and then .cpp.elf

[Reply #10 on:]

Hi liudr, the Arduino IDE 1.0 does not make it easy to get the avr-size info.
In contrast, Microchip MPLAB has a menu selection "View > Memory Usage Gauge".
What I did with IDE 1.0 was:

1. change "File > Preferences > preferences.txt" to say "build.verbose=true".

2. looked at the IDE message window to find where the build files are stored
   during compile. The path is VERRRRY long with many subdirectories,
   c:\DOCUM~1\userme\LOCALS~1\Temp\build37068458739447598749.tmp. There I
   found the .elf build file.

3. then I moved both avr-size.exe [found deep in the Arduino IDE directory
   and the [renamed] .elf file up to c:\, where I ran the command shown from
   "Start > Run > cmd", and "cd \".

4. finally, I got the info shown.

This info is very helpful to me, as you can see, my sketch is already using much
of the mega chip resources, and the sketch is only about half done. Also, the stats
are shown for having all of the troublesome menu strings commented out, but now I
can investigate the effect of adding the menuing back in.

Actually, I don't much care for the problems involved with string-literals >> RAM,
and use of the rather cumbersome PROGMEM constructs. Since my controller board has
a 24LC256 I2C eeprom onboard, I am writing code to store all of the menu strings
there, then mega storage and RAM space are no longer a problem.

Well, I've learned about enough in the past couple of days to be really dangerous
now. LOL.

"A little knowledge is not so much dangerous as useless."
- Jim Harrison, in 'The Beast That God Forgot to Invent'.

[Reply #11 on:]

This is what I found with my latest mucking around:

====================
C:\avr-temp>avr-size -C --mcu=atmega328p svt2-1.elf
AVR Memory Usage
----------------
Device: atmega328p
Program:   15934 bytes (48.6% Full)
(.text + .data + .bootloader)
Data:       1664 bytes (81.3% Full)
(.data + .bss + .noinit)


C:\avr-temp>avr-size -C --mcu=atmega328p svt2-2.elf
AVR Memory Usage
----------------
Device: atmega328p
Program:   16538 bytes (50.5% Full)
(.text + .data + .bootloader)
Data:       1984 bytes (96.9% Full)
(.data + .bss + .noinit)


C:\avr-temp>avr-size -C --mcu=atmega328p svt2-3.elf
AVR Memory Usage
----------------
Device: atmega328p
Program:   17144 bytes (52.3% Full)
(.text + .data + .bootloader)
Data:       2590 bytes (126.5% Full)
(.data + .bss + .noinit)
=============================

The first size test is for the sketch that worked properly yesterday after I commented out most
of the menuing strings.

The 2nd size test is the sketch with the menuing strings re-inserted back into the program. This is
the program that crashed.

The 3rd size test is where I added a bunch of additional strings to the menus. It's obvious here
that the data space is far exceeded. 126% Full. However, the IDE did NOT issue any error indications
during the compile.

Well, I guess this solves the entire matter that I brought up in the first place.