Not enough memory: Options needed.

My sketch stopped working when my ATmega328 ran out of memory for variables. So I'm looking for alternatives to storing some of the data.

One big single chunk of memory is consumed by the following code:

struct soundhdr {
 char  riff[4];        /* "RIFF"                                  */
 long  flength;        /* file length in bytes                    */
 char  wave[4];        /* "WAVE"                                  */
 char  fmt[4];         /* "fmt "                                  */
 long  block_size;     /* size of FMT chunk in bytes (usually 16) */
 short format_tag;     /* 1=PCM, 257=Mu-Law, 258=A-Law, 259=ADPCM */
 short num_chans;      /* 1=mono, 2=stereo                        */
 long  srate;          /* Sampling rate in samples per second     */
 long  bytes_per_sec;  /* bytes per second = srate*bytes_per_samp */
 short bytes_per_samp; /* 2=16-bit mono, 4=16-bit stereo          */
 short bits_per_samp;  /* Number of bits per sample               */
 char  data[4];        /* "data"                                  */
 long  dlength;        /* data length in bytes (filelength - 44)  */
} wavheader;

void setup()
{
 strncpy(wavheader.riff,"RIFF",4);
 wavheader.flength = 10000000; // file length in bytes
 strncpy(wavheader.wave,"WAVE",4);
 strncpy(wavheader.fmt,"fmt ",4);
 wavheader.block_size = 16; // Size of above header
 wavheader.format_tag = 1; // PCM
 wavheader.num_chans = 1; // channels 1=mono, 2=stereo
 wavheader.srate = 19500;  // Sampling rate in samples per second
 wavheader.bytes_per_sec = 19500; // bytes per second = srate*bytes_per_samp
 wavheader.bytes_per_samp = 1; // 2=16-bit mono, 4=16-bit stereo
 wavheader.bits_per_samp = 8;  // Number of bits per sample
 strncpy(wavheader.data,"data",4); // "data"
 wavheader.dlength = 10000000; // data length in bytes (filelength - 44)
}

(1) I suspect that the wavheader struct itself, AND the setup() lines that fill it with data, BOTH take up about 45 bytes of SRAM memory, thus consuming over 90 bytes of SRAM total? (Or perhaps the lines in setup() are totally in program Flash memory, which isn't as bad.)

(2) This structure is really a constant, not a variable, in that it never changes. It is used as the "header" in creating new .WAV audio files. So I believe there should be a way to store it as one complete 45-byte object, instead of first creating the structure, and then adding the lines of code that fill that structure. Storing as a completed structure should cut memory used in half.

(3) Storing it as part of the program with "PROGMEM" wouldn't be bad, except that requires adding the library

#include <avr/pgmspace.h>

And special functions to store this structure and read it back. All of which sounds like using more memory than I would be saving.

(4) And then there's EEPROM memory, which sounds like maybe the best idea. I've got 1024 bytes of that just waiting to be put to a good use.


So really, I have three questions about all the above:

  1. What is the best place to store large 'constants' like this?
  2. How do I make this structure a completed constant I can store, instead of it being an empty structure I reload with data each time?
  3. How do I place this loaded structure into permanent memory, and retrieve it when needed for new file creation?

Tommy

Post your code.

Pick up a Bobuino2 from Crossroads.
Lots of RAM
Lots of memory.
Lots of I/O
Comes with terminals, SD CARD, RTC, RS232
Compatable.

And a 25 minute air supply for under water work.
.

45 bytes out of nearly 2048 may not be the problem. If it is, you are too close to the edge anyway.
IIRC you need that header to either read or write a WAV file, so it can't all go in flash because the filesize and some of the other info in the structure changes.

Like AWOL says, post your code.

Pete

You must post all and I do mean all your code to get any real help

Mark

I don't think posting all my code would help. My project uses three ATmaga328 chips communicating with each other via SoftwareSerial. So there are three large sketches; one for video, one for audio, and one for everything else. A single sketch would make little sense by itself; and I've been previously told posting more was "nearly incomprehensible". So it sounds like I'm kinda outa luck here.

I don't understand why giving one complete example of a memory-using object and asking how to tuck it away elsewhere, is not enough info. If I understood how to deal with that one, I could use the same techniques on the other large objects in memory by myself.

get more memory, the DUE would work.
You can't tuck memory somewhere else. You can tuck static data on an SD card or something.

CosmickGold:
(1) I suspect that the wavheader struct itself, AND the setup() lines that fill it with data, BOTH take up about 45 bytes of SRAM memory, thus consuming over 90 bytes of SRAM total? (Or perhaps the lines in setup() are totally in program Flash memory, which isn't as bad.)

No, the 'struct' itself doesn't use any SRAM, but the declared variable does (44 bytes).

In addition, all your strings you are using during the setup() function like ,"RIFF", "WAVE" or "fmt " are using additional SRAM.

If you need an initialized variable perhaps better do it like that:

struct soundhdr {
 char  riff[4];        /* "RIFF"                                  */
 long  flength;        /* file length in bytes                    */
 char  wave[4];        /* "WAVE"                                  */
 char  fmt[4];         /* "fmt "                                  */
 long  block_size;     /* size of FMT chunk in bytes (usually 16) */
 short format_tag;     /* 1=PCM, 257=Mu-Law, 258=A-Law, 259=ADPCM */
 short num_chans;      /* 1=mono, 2=stereo                        */
 long  srate;          /* Sampling rate in samples per second     */
 long  bytes_per_sec;  /* bytes per second = srate*bytes_per_samp */
 short bytes_per_samp; /* 2=16-bit mono, 4=16-bit stereo          */
 short bits_per_samp;  /* Number of bits per sample               */
 char  data[4];        /* "data"                                  */
 long  dlength;        /* data length in bytes (filelength - 44)  */
};

soundhdr wavheader={
 {'R','I','F','F'},        /* "RIFF"                                  */
 10000000, // file length in bytes
 {'W','A','V','E'},
 {'f','m','t',' '},
 16, // Size of above header
 1, // PCM
 1, // channels 1=mono, 2=stereo
 19500,  // Sampling rate in samples per second
 19500, // bytes per second = srate*bytes_per_samp
 1, // 2=16-bit mono, 4=16-bit stereo
 8,  // Number of bits per sample
 {'d','a','t','a'}, // "data"
 10000000, // data length in bytes (filelength - 44)
};

void setup(){
}


void loop() {
}

When doing so, the variable 'wavheader' will be initialized before setup even begins executing, and you will not waste additional RAM in constant strings.

BTW: Why should 44 bytes out of 2048 bytes be a problem in your sketch? What about the other 2004 bytes?

Using SD library will cost you more than 1 KB (>1024) for handling the file system and SD card only! So maybe if your sketch needs file handling and big amounts of SRAM, you possibly need a microcontroller board which provides more RAM as an UNO (MEGA perhaps).

See Gammon Forum : Electronics : Microprocessors : Putting constant data into program memory (PROGMEM)

That shows exactly how to put constant data into PROGMEM. I use that, for example, in sketches which have lots of Arduino board types in them (the signature, the board name, the amount of RAM, etc.).

jurs, you've done a perfect job of answering question #2. Beautiful. Eloquent. I've tested your code and put it into use already.

Nick, thank you for providing the answer to questions #1 and #3. I've been looking over how you explained it on your website, trying to decide the best approach.

mistergreen, I think you pointed out a real problem when you said, "You can't tuck memory somewhere else." Thinking about that, I've concluded it means that even if I stored a structure somewhere else like in PROGMEM or EEPROM, I've still got to pull it out into regular memory before I can use it; so it might as well have been in regular memory the whole time anyway, nothing saved. Bummer, but thanks for pointing that out.

CosmickGold:
if I stored a structure somewhere else like in PROGMEM or EEPROM, I've still got to pull it out into regular memory before I can use it; so it might as well have been in regular memory the whole time anyway, nothing saved. Bummer, but thanks for pointing that out.

Not the whole thing though..

You can store the whole thing in EEPROM/PROGMEM and read the individual parts you need. There is no need to read the whole thing unless you need it.

But you are doing what you think would be best (X/Y ??), however it may be better to look at how you can reduce the overall memory usage, not just move it around. As jurs pointed out: BTW: Why should 44 bytes out of 2048 bytes be a problem in your sketch? What about the other 2004 bytes?

For this we need code.

BTW: Why should 44 bytes out of 2048 bytes be a problem in your sketch? What about the other 2004 bytes?

Really? I thought it was normal for those 2048 bytes to disappear pretty quickly -- apparently absorbed by libraries and such -- because it always seems to happen to me.

If that's not normal, I'd love to get some of those bytes back.

Here's the sketch this post has been about. I assumed I was out of SRAM when the line

entry =  dir.openNextFile();

failed to find even one file, and removing a few global variables anywhere made it start working again.

Currently, there are probably around 200 bytes of free SRAM, because I've reduced the size of the lines

byte buffA[32]; // 256 // 128
byte buffB[32];

from 128 to 32 bytes each; which of course, has made everything work again.

But where have the other 1848 bytes gone? Is there an easy way to figure that out?

PS. I got an alert that read: "The message exceeds the maximum allowed length (9000 characters)."
So I had to make the sketch an attachment.

DM_Waves.ino (9.51 KB)

    Serial.println(F("SD Card failed to start!"));
    err = 1;
  }
  else
  {
    Serial.println(F("SD Card started."));
  }

There's a few more bytes clawed back

loadBuffer("2.The file named ", fName, " was not found");

More wasted RAM

Edit: My guess is that around 12-15% of your RAM is wasted (assuming this is a Uno with 2048 bytes of RAM)

Thanks for the lesson! I didn't know about F(). But googled your example which led me to:
You can pass flash-memory based strings to Serial.print() by wrapping them with F().

Now I can go through my code and F() the other strings as well. :slight_smile:

CosmickGold:
But where have the other 1848 bytes gone? Is there an easy way to figure that out?

Several bytes are used by libraries.

For example: Each "Serial" used will cost you more than 128 bytes: 64 bytes for the serial input buffer and 64 bytes for the serial output buffer.

The "SD" library will cost you a huge amount of RAM. I think 1 kB RAM is easily gone: One 512 bytes FAT sector to hold the file allocation table, and additional 512 bytes for each file open at the same time.

But you are also wasting RAM in your sketch in "string constants", such like:

 Serial.print("2.Got Command: ");

This string is printed from RAM. You could better do when printing string constants from flash directly, using the F-macro:

 Serial.print(F("2.Got Command: "));

In that case, the string will not need additional RAM, but be printed from flash (program) memory directly.

So perhaps one thing to save RAM would be: Watch out for every "print" and "println" in your code. If the parameter is a constant string, then add the F-macro like shown above!

loadBuffer("2.The file named ", fName, " was not found");

This could (should!) be replaced by

Serial.print (F("2.The file named "));
Serial.print (fName);
Serial.println (F(" was not found"));

CosmickGold:
PS. I got an alert that read: "The message exceeds the maximum allowed length (9000 characters)."
So I had to make the sketch an attachment.

That's what attachments are for.

How to use this forum

CosmickGold:
Currently, there are probably around 200 bytes of free SRAM, because I've reduced the size of the lines...

You can print the current free ram in the middle of your code. It can change a little, depending on where you place it. The minimum will probably be inside a library, like SD, and so it's harder to find that out.

I added a called to "freeMemory()" and it printed 555 bytes within setup() and 556 within loop(). SD, at least the 1.0.5 IDE version, requires about 300 free bytes to operate. That's consistent with your determination that you effectively have about 200 free bytes.

SdFat uses a little less RAM (total of about 650 bytes) than SD and it also uses less stack. It's not a huge savings though, something around 120 bytes combined. Maybe that's worth the trouble of changing libraries? I haven't compared the latest version of SdFat against the lastest version of SD so I'm not sure what you'd get if you switched.

I tried inserting a different version of software serial. It didn't save that much RAM, something like 20 bytes. It did reduce the code size by ~1K, but that's not where your problem is.

What a wealth of valuable information, from EACH of you guys!

I'm truly delighted!

Now I'm off to implement all the above suggestions; and with a much clearer understanding of what's happening, and what I'm doing. :slight_smile: