Problem with large-ish arrays of strings

system · September 22, 2014, 5:47am

Hey everyone,
I'm starting a little project using an arduino diecimilia, displaying on a screen a Resistor value and reference.

I've started out by creating some arrays and displaying them through Serial.print and I've run into some very strange trouble. Depending on the data in the array, the output will either be totally fine, or completely corrupt or non-existant.

My arrays have about 100 items, and look like this:

const char* refs[] = { 
	"R30",
	"R188",
       (ETC)
	"R190",
	"R189"
};
const char* vals[] = { 
	"10K",
	"470",
	"1K",
        (ETC)
	"960"
};

and the loop part has just a for loop, displaying things like:

    Serial.print(refs[a]);
    Serial.print("\t");
    Serial.print(vals[a]);

I'm wondering if I'm reaching a point of adding too many variables, or taking up too much memory and it's starting to mess up?

system · September 22, 2014, 5:52am

attached is the full code. I’m using Arduino 1.0.5

This code works totally fine and displays sweet in the Serial Monitor. But if you add one more array item under refs then it will break.

stuffomatic1.ino (1.91 KB)

system · September 22, 2014, 6:01am

sounds like your running out of SRAM where the strings/variables are being stored unless you declare them in program memory

it should solve your problem this time around

system · September 22, 2014, 6:48am

It’s pointless attaching such a small amount of code, just post it between code tags.

const int dataRows = 8; 

const char* refs[] = { 
	"REF",
	"",
	"R30",
	"R188",
	"R190",
	"R189",
	"R117",
	"R9",
	"R6",
	"R2",
	"R5",
	"R4",
	"R1",
	"",
	"R126",
	"R133",
	"R130",
	"R124",
	"R122",
	"R109",
	"R95",
	"R96",
	"R94",
	"R105",
	"R106",
	"R108",
	"R107",
	"R97",
	"R99",
	"R98",
	"R100",
	"R110",
	"R125",
	"R129",
	"",
	"R132",
};
const char* vals[] = { 
	"VAL",
	"",
	"10K",
	"470",
	"1K",
	"960",
	"10K",
	"3K",
	"100K",
	"470K",
	"1M",
	"22K",
	"470K",
	"",
	"1K",
	"100K",
	"18K",
	"10K",
	"100K",
	"1.5M",
	"100K ",
	"100K",
	"100K ",
	"47",
	"820",
	"820",
	"47",
	"100K",
	"100K",
	"100K",
	"2.4M",
	"1.5M",
	"100K",
	"10K",
	"18K",
	"100K",
	"1K",
	"",
	"390",
	"2.2K",
	"2.2K",
	"1K",
	"150K",
	"18K",
	"22K",
	"18K",
	"18K",
	"18K",
	"18K",
	"22K",
	"18K",
	"150K",
	"1K",
	"2.2K",
	"2.2K",
	"390",
	"",
	"20K",
	"10K",
	"10K",
	"10K",
	"",
	"",
	"2.2K",
	"150K",
	"150K",
	"100K",
	"1K",
	"470",
	"470",
	"470",
	"470",
	"1K",
	"100K",
	"150K",
	"150K",
	"2.2K",
	"",
	"",
	"10K",
	"10K",
	"10K",
	"20K",
	""
};
const char* notes[] = { 
	"NOTES",
	"START PASS A",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"ROW 2",
	"",
	"",
	"",
	"",
	"",
	"VERTICAL",
	"MATCHED 1",
	"",
	"MATCHED 1",
	"",
	"",
	"",
	"",
	"MATCHED 2",
	"",
	"MATCHED 2",
	"",
	"VERTICAL",
	"",
	"",
	"",
	"",
	"",
	"ROW 3",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"ROW 4",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	".2 OR .3 ONLY",
	"",
	"",
	"",
	"",
	"",
	"",
	".2 OR .3 ONLY",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"",
	"ROW 5"
};





void setup() {

  Serial.begin(9600);

}


void loop () {

  for (int a = 0; a < dataRows; a++) {
    Serial.print("\t");
    Serial.print(a);
    Serial.print("\t");
    Serial.print(refs[a]);
    Serial.print("\t");
    Serial.print(vals[a]);
    Serial.print("\t");
    Serial.print(notes[a]);
    Serial.print("\n");
    delay (100);
  }

}

system · September 23, 2014, 1:03am

ardiri:
sounds like your running out of SRAM where the strings/variables are being stored unless you declare them in program memory

http://arduino.cc/en/Reference/PROGMEM

it should solve your problem this time around

Thanks ardiri, I'll try this tonight. Does anyone know of any examples that show how to store and retreive a 2-dimensional array in PROGMEM?

Mike

econjack · September 23, 2014, 2:06am

Does anyone know of any examples that show how to store and retreive a 2-dimensional array in PROGMEM?

Check out the answer, especially the two sources in the second paragraph of the answer posed on:

They might be helpful.

system · September 23, 2014, 7:25am

Thanks for the help so far, I've gotten it working in PROGMEM and now I'm using a Teensy 2.0 for some more memory.

Declaring strings in PROGMEM seems a bit labour intensive?

prog_char string_0[] PROGMEM = "String 0";   // "String 0" etc are strings to store - change to suit.
prog_char string_1[] PROGMEM = "String 1";
prog_char string_2[] PROGMEM = "String 2";
prog_char string_3[] PROGMEM = "String 3";
prog_char string_4[] PROGMEM = "String 4";
prog_char string_5[] PROGMEM = "String 5";


// Then set up a table to refer to your strings.

PROGMEM const char *string_table[] = 	   // change "string_table" name to suit
{   
  string_0,
  string_1,
  string_2,
  string_3,
  string_4,
  string_5 };

I can generate it all with a script, so I guess that's not a big deal, but is there a way to at least save all those const declarations in a separate file to clean it up a bit? Sorry for all the noob questions guys,

system · September 23, 2014, 9:03am

but is there a way to at least save all those const declarations in a separate file

Sure. You can add a new tab, and name the file with a .h extension, and then use a #include statement to include the header file.

system · September 28, 2014, 10:05pm

PaulS:

but is there a way to at least save all those const declarations in a separate file

Sure. You can add a new tab, and name the file with a .h extension, and then use a #include statement to include the header file.

Perfect, I couldn't get the .h thing to work but it works fine as a separate .ino file as long as that tab with the variable declarations is before the .ino tab with the program.

I'm generating the complicated PROGMEM declarations with a php script that takes a .csv and I just paste the output into the new tab. I also moved to a Teensy 2.0 for more memory as well.. thanks for all the help guys.

nickgammon · September 29, 2014, 3:34am

You’ve got me interested now. There must be a better way of doing this than having huge lots of variables, whose name you don’t care about, which are then referenced elsewhere.

So I experimented a bit …

#include "memdebug.h"

#define USE_PROGMEM true

typedef struct {
   char description [12];
} descriptionType;

descriptionType 
#if USE_PROGMEM
PROGMEM
#endif
refs[120] = { 
           { "Twas" },
           { "bryllyg" },
           { "and" },
           { "ye" },
           { "slythy" },
           { "toves" },
           { "Did" },
           { "gyre" },
           { "and" },
           { "gymble" },
           { "in" },
           { "ye" },
           { "wabe" },
           { "All" },
           { "mimsy" },
           { "were" },
           { "ye" },
           { "borogoves" },
           { "And" },
           { "ye" },
           { "mome" },
           { "raths" },
           { "outgrabe" },
           { "Beware" },
           { "the" },
           { "Jabberwock" },
           { "my" },
           { "son" },
           { "The" },
           { "jaws" },
           { "that" },
           { "bite" },
           { "the" },
           { "claws" },
           { "that" },
           { "catch" },
           { "Beware" },
           { "the" },
           { "Jubjub" },
           { "bird" },
           { "and" },
           { "shun" },
           { "The" },
           { "frumious" },
};

// number of items in an array
template< typename T, size_t N > size_t ArraySize (T (&) [N]){ return N; }


void setup ()
  {
  Serial.begin (115200);
  Serial.println ();
  Serial.print (F("Free memory = "));
  Serial.println (getFreeMemory ());

  for (int i = 0; i < ArraySize (refs); i++)
    {
    Serial.print (F("i = "));
    Serial.print (i);
    Serial.print (F(", word = "));  
    descriptionType oneItem;
#if USE_PROGMEM    
    memcpy_P (&oneItem, &refs [i], sizeof oneItem);
#else
    memcpy (&oneItem, &refs [i], sizeof oneItem);
#endif
    Serial.println (oneItem.description);
    }   // end of for loop

  }  // end of setup

void loop () { }

Output:

Free memory = 1686
i = 0, word = Twas
i = 1, word = bryllyg
i = 2, word = and
i = 3, word = ye
i = 4, word = slythy
i = 5, word = toves
i = 6, word = Did
i = 7, word = gyre
i = 8, word = and
i = 9, word = gymble
i = 10, word = in
i = 11, word = ye
i = 12, word = wabe
...

The important thing is that there are 1686 bytes of RAM free. Compare this to a sketch that just reports free memory:

#include "memdebug.h"

void setup ()
  {
  Serial.begin (115200);
  Serial.println ();
  Serial.print (F("Free memory = "));
  Serial.println (getFreeMemory ());
  }  // end of setup

void loop () { }

Output:

Free memory = 1702

So we only lost 16 bytes of RAM (1702 - 1686), so clearly neither the array itself nor the words in it, have been copied to RAM.

Now if we change it to not use PROGMEM:

#define USE_PROGMEM false

Output:

Free memory = 244
i = 0, word = Twas
i = 1, word = bryllyg
i = 2, word = and
i = 3, word = ye
i = 4, word = slythy
i = 5, word = toves
i = 6, word = Did
i = 7, word = gyre
i = 8, word = and
i = 9, word = gymble
i = 10, word = in
i = 11, word = ye
i = 12, word = wabe
...

Now we have definitely lost RAM. 1442 bytes of it (1686 - 244). Now since we have 120 array items of 12 bytes each, that sounds right (120 * 12 = 1440).

This method constrains you to decide in advance how large the strings are going to be (I chose 12 bytes each) but in the example above that would be plenty. And even the code above has plenty of PROGMEM over:

Binary sketch size: 4,290 bytes (of a 32,256 byte maximum)

Now that we have got this far, we can have more stuff, for example a description, meaning and some sort of value.

typedef struct {
   char description [12];
   char meaning [20];
   int value;
} descriptionType;

descriptionType PROGMEM refs[120] = { 
           { "Twas",    "it was",       42 },
           { "bryllyg", "four o'clock", 18 },
           { "and",     "whatever",     33 },
           { "ye",      "the",          19 },
           { "slythy",  "slippery",    100 },
           { "toves",   "badgers",    1000 },

};

// number of items in an array
template< typename T, size_t N > size_t ArraySize (T (&) [N]){ return N; }

void setup ()
  {
  Serial.begin (115200);
  Serial.println ();

  for (int i = 0; i < ArraySize (refs); i++)
    {
    Serial.print (F("i = "));
    Serial.print (i);
    Serial.print (F(", line = "));  
    
    // make a copy of the current one
    descriptionType oneItem;
    memcpy_P (&oneItem, &refs [i], sizeof oneItem);
    
    Serial.print (oneItem.description);
    Serial.print (F(", "));
    Serial.print (oneItem.meaning);
    Serial.print (F(", "));
    Serial.println (oneItem.value);
    }  // end of for loop

  }  // end of setup

void loop () { }

Output:

i = 0, line = Twas, it was, 42
i = 1, line = bryllyg, four o'clock, 18
i = 2, line = and, whatever, 33
i = 3, line = ye, the, 19
i = 4, line = slythy, slippery, 100
i = 5, line = toves, badgers, 1000
...

system · May 5, 2021, 8:53pm