Using up 724% memory when accessing multidimensional array

Hi,

I'm guessing that there is something to do with how array's access data here that I haven't yet quite figured out with pointers and/or dereferencing. I think I understand how those are meant to work - but can't get it to work :o .

I can work with the data and the compiler doesn't throw an error. But as soon as I try a Serial.println, it throws the memory usage way out of scope.

My understanding is that when I do a sizeof( gameGrid[20][20].designation), I get "1" because that is the size of the first character of the array within it. But can't get it to tell me how long the whole series of characters is - such that I can confirm that I haven't looped in 13MB of characters.

I see that the code "shrinks" when I remove items from the tileData struc (well duh) - but it removes megabytes of data if I leave only the designation within it. Those are all simply initialized variables. Is it initializing the whole grid all of these items? Even when I comment out everything but the designation, it takes up 1796 bytes

The code is as follows:

const char myDesignation = 'H';

// Grid
const int GRID_X = 40;
const int GRID_Y = 40;
const int GRID_OFFSET = 2;


typedef struct {
  unsigned long serialNumber;
  char designation;
  byte neighbor;

} tileData;

tileData gameGrid[GRID_X][GRID_Y];                             // Create game grid bucket
int gameGridOffsetX = GRID_X/GRID_OFFSET;               // This is the number that is our actual zero point of the game grid - allowing for placement to the left of other tiles.
int gameGridOffsetY = GRID_Y/GRID_OFFSET;

void setup() {
  Serial.begin(9600);
  Serial.println("Startup");

  gameGrid[20][20].designation = myDesignation;

}

void loop() {

  Serial.println(gameGrid[20][20].designation);
  //char check = gameGrid[20][20].designation;
  //int *deep = sizeof(gameGrid[20][20].designation);
 //Serial.println(*deep);  
   // Comes up with 256 - so I feel like I'm overloading the buffer.



  delay(100);
}

Output:
Sketch uses 1664 bytes (5%) of program storage space. Maximum is 30720 bytes.
Global variables use 9796 bytes (478%) of dynamic memory, leaving -7748 bytes for local variables. Maximum is 2048 bytes.

Incidentally (another post?): why do I have to assign " gameGrid[20][20].designation = myDesignation;" within a function? Why won't it assign beneath int gameGridOffsetY = GRID_Y/GRID_OFFSET;? It says something about its type not being declared.

Well, each instance of your tileData type occupies 6 bytes (4+2+2).

So, 6 x 40 x 40 = 9600 bytes. Your board has 2048 bytes of memory.

Do you see a problem?

EDIT:
If you never actually access the array that you define, chances are it's completely removed by compiler optimizations.

To fit this thing into available memory, you can dramatically reduce the amount of storage required.

For example, what is "serialNumber" and why should it be a long integer? There are only 1600 elements in a 40x40 array.

You don't need a byte to index 256 neighbors. 4 bits will identify 16 neighbors. The other 4 in that byte could be used for "my designation", if 16 possibilities would suffice.

judging by your code i assume you are coming from desktop or mobile programming. the ram on these boards are very very small. arduous programming teaches a great lesson in proficiency. You're going to have to reconsider your approach.

SD card readers for arduino are very inexpensive. An alternative would be to pull data on and off an SD card as needed instead of storing it in the RAM.

My understanding is that when I do a sizeof( gameGrid[20][20].designation), I get "1" because that is the size of the first character of the array within it.

You understand wrongly. sizeof(gameGrid[20][20]) gives you the number of bytes used by the data at the 21st location in each direction, which is a shame because the array only has 20 dimensions in each direction. What you are doing by using .designation I have no idea

If you want to know how many bytes an array uses then use

sizeof(gameGrid)

UKHeliBob:
You understand wrongly. sizeof(gameGrid[20][20]) gives you the number of bytes used by the data at the 21st location in each direction, which is a shame because the array only has 20 dimensions in each direction. What you are doing by using .designation I have no idea

gameGrid [20][20].direction is the direction element of the structure stored in the specified element of the array, direction is declared as a char, so size is 1 byte.

Sorry about misleading you. I was certain that gameGrid was declared as a 20 by 20 array but I don't know where I got that idea from

gfvalvo:
Well, each instance of your tileData type occupies 6 bytes (4+2+2).

So, 6 x 40 x 40 = 9600 bytes. Your board has 2048 bytes of memory.

Do you see a problem?

EDIT:
If you never actually access the array that you define, chances are it's completely removed by compiler optimizations.

Hmm. Ok. I'm having a "well duh" moment here. I thought I did that math. I just forgot that all that space is taken up upon initialization.

But I'm still curious why it only blows up when I use the Serial.println()? If I remove that, the program compiles.

Thank you

jremington:
To fit this thing into available memory, you can dramatically reduce the amount of storage required.

For example, what is "serialNumber" and why should it be a long integer? There are only 1600 elements in a 40x40 array.

You're right. I can remove the serial number for the program I am making. I am creating something that will have tens of thousands of units available, eventually. But if I can ditch irrelevant data, then fantastic!

jremington:
You don't need a byte to index 256 neighbors. 4 bits will identify 16 neighbors. The other 4 in that byte could be used for "my designation", if 16 possibilities would suffice.

I like your idea of using the rest of the byte as something else! I am currently pushing around letters of the alphabet. And only have to track four neihbors. I hadn't come across the bit operators before when asking myself "I wonder if I can just use a bit? There doesn't seem to be a 'bit' type...") and was just running around with hunks of program taking up useless space. (Wow.) You've helped me ask the right question and find the right thing.

Edit: Now I remember I was trying to keep things simple and be proto only guy and bring in experts for final code. But learning bitwise and bitRead() operations might make me better at the proto part, too.End Edit

My question then might become "How much processing power gets used in moving bits around to save on storage?" (I'll do some googling on that).

Great points. Thanks a bunch!

taterking:
judging by your code i assume you are coming from desktop or mobile programming. the ram on these boards are very very small. arduous programming teaches a great lesson in proficiency. You're going to have to reconsider your approach.

SD card readers for arduino are very inexpensive. An alternative would be to pull data on and off an SD card as needed instead of storing it in the RAM.

Valid points. I'm going to streamline to see if I can put everything that is code onto the Nano.

Cheers

david_2018:
gameGrid [20][20].direction is the direction element of the structure stored in the specified element of the array, direction is declared as a char, so size is 1 byte.

Understood. I was able to test for that.

Allister_McRae:
Hmm. Ok. I'm having a "well duh" moment here. I thought I did that math. I just forgot that all that space is taken up upon initialization.

But I'm still curious why it only blows up when I use the Serial.println()? If I remove that, the program compiles.

Thank you

No, the space is taken up when the compiler realizes that you're actually going to use the variable that you defined. Before for you tried to print an element of the array, the compiler realized you weren't using it so optimized it away (just as I said in my Reply #1)

gfvalvo:
No, the space is taken up when the compiler realizes that you're actually going to use the variable that you defined. Before for you tried to print an element of the array, the compiler realized you weren't using it so optimized it away (just as I said in my Reply #1)

Now I understand that the println is what "makes it real." Thank you.

If you get or build an ATmega1284 duino you will have 16KB RAM.

A Mega2560 Arduino can host external RAM though the internal RAM is 8KB.

Many ARM chip duinos have 32KB RAM or more. Arduino Due is one such board, Teensy 3.1 to 4.0 are others.

Smart thing is more likely connect an SD module unless you need blazing speed.

GoForSmoke:
Smart thing is more likely connect an SD module unless you need blazing speed.

Thats probably the easiest for get'er'done style prototyping with my level of knowledge.

But I'm so close to figuring out bitmasking (I think). I really need to find professionals for these types of things ;).

Thanks for the input!

Have you seen the bit math tutorial in the Arduino Playground?
https://playground.arduino.cc/Code/BitMath/

In decimal math 1234 is 1x1000 + 2x100 + 3x10 + 4x1.

In binary math the same value is 0b10011010010 (0b shows the number is binary) for

1x1024 + 0x512 + 0x256 + 1x128 + 1x64 + 0x32 + 1x16 + 0x8 + 0x4 + 1x2 + 0x1

Remember the bundles of pencils way back in 1st and 2nd grade?
A binary bundle has 1 pencil. It's simpler than 10, but not what we're used (programmed) to.

Take care to not get an SD module that uses shortcut voltage leveling (I bought some in 2012) as they reduce the lifetime of the SD card. There are cheap SD modules with proper leveling (I am told) just not ultra-cheap ones.

Also the new Nano boards with 33 in the name have loads of RAM, the BLE version runs $19.95.

Added: You could also get a 1284P DIP for $5.50 and build a duino. That AVR has 16KB RAM. The Nick Gammon minimal breadboard duino tutorial covers the 328P and "The Mighty" 1284P as examples. I have some 1284Ps and Nick's tutorial got me through building, bootloading and running them.