"Load mapping" SRAM memory use in Arduino?

First, I'm a geologist, not an EE guy; I can do simple sketches, but this stuff is way over my head.

In short, I'm using the Arduino Pro in an electronic flow controller for educational river models we build (see www.emriver.com.

A pro programmer wrote some nice code for me as a favor (in C++); it works, but we're bumping against memory limits; we have several large arrays, most of which are used to run "hydrographs" -- simulation of a flood; a curve that starts small, climbs to a max, and then tapers off.

I am cleaning up and refining things because he's already done me WAY too many favors.

Also we had to hard code a pwm v. flow rate curve because it could not be well described by a math function; so 255 numbers right there (we unit8_t 'd them, that helped).

Also he as advised using "const" for variables, this put them in flash, correct?

Many thanks for help

My friend isn't much versed on Arduino; he's asked me to ask you guys if there is a way to "load map" with the Arduino IDE, i.e. see how much SRAM is being used when the program runs.

I assume you're using a 328 chip.

I had a brief search for ATMega emulators but emulare was the only one I found that looks usable without a lot of work. Unfortunately it didn't seem very stable when I used it. Your mileage may vary.

What use are you making of flash, EEPROM and RAM? There are techniques that you can use to trade off between them, but that's no good if they're all maxing out.

Do you understand how your RAM is being used i.e. what is taking up the space? You can fiddle around with low level details of the storage mechanisms, for example you might be able to use less than 255 entries for your PWM calibration curve, but the biggest gains usually come from correcting architectural mistakes. For example, why are you using C++ rather than 'C'? Do you have much dynamic data? Are you using dynamic buffers anywhere? It's easy to grossly increase heap consumption just by misusing strings.

I don't know what the memory layout is in the Arduino runtime, but usually the memory use will consist primarily of heap and stack. Do you know how much space each of these is using? Stack use you can determine from the address of an item on the top of the stack. heap use is harder to do and probably requires diving into the runtime library, but since it's open source that should be possible.

Thanks PeterH.

Using the Pro 328. We have tons of flash left over, the code's only about 7K; it's the large arrays and lookup tables; my friend has used pointers for some as a help (as I understand it).

Part of the problem is that I want the code to be hackable by students using our models (from grade 5 on up to PhD); and the code is now still barely accessible because it's pretty complex.

And I think my friend did use C, I don't really know the difference, just that it compiles and runs fine, sorry, again, I'm a geoscientist trying to do the jobs of several people here -- we are struggling in this economy.

I did find info on PROGMEM: Arduino Playground - Memory

Looks like this would work, but with added complexity.

Also this is a production item, we'll send dozens of these out to clients, and anything that risks flaky behavior is a real risk for us; most users will not want to reprogram themselves.

Would really help if we could just calculate loading of sram, but I don't see a simple way to do that.

PeterH,

Sorry, should have mentioned I can email your replies to my friend and this may be a big help to him -- he'll know what all that means; he's a high level pro but not familiar with the specifics of Arduino.

He wanted to start from scratch but I wanted to use Arduino to make the project open source and easily modifiable by users. No good deed goes unpunished, right?

What does your Arduino actually do?

It sounds as if it is encompassing some pretty significant logic.

Have you considered changing the system architecture so that the Arduino acts as a remote sensor/actuator interface and the main logic happens on a PC? By providing a COM interface to the Arduino you can implement all the PC side stuff in Windows Script Host which people can code / hack in VB or javaScript. I'd consider that far more approachable for people who want to dive into the system. It also means the part they're hacking is just a file on the PC so easily fixable even by a non-techy. (And it means it is easy to produce log files, access databases and so on and enables all sorts of interesting possibilities. It does mean that you need a PC/laptop as well as the Arduino, and I don't know if that would be an issue, but it seems these days you can usually assume there is a computer wherever there is a person.

There are a number of free memory functions that'll tell you how much SRAM you're consuming - search the forum for freemem.

An easy way out if you're short of space is to upgrade to a Mega - it's more expensive, but perhaps not so bad when that cost is wrapped up with the other components in your product.

I'd assumed as you're making a commercial product that your code is proprietary, but you mention that it's open source. In which case, post it here and there may be things the folks here can suggest to improve/clarify/reduce the memory footprint.

What part does the arduino play in the system? Flow control by PWMing the pump?

Can you post the code you have at the moment?

A pro programmer wrote some nice code for me as a favor (in C++); it works, but we're bumping against memory limits; we have several large arrays, most of which are used to run "hydrographs" -- simulation of a flood; a curve that starts small, climbs to a max, and then tapers off.

How large are these arrays? Can you post such a hydroGraph?

For these kind of problems approximation by interpolation can be a real memory winner, I wrote the multimap() function for these kind of problems. see - Arduino Playground - MultiMap - The idea of multimap() is to reduce all points of an array or a function (the wave in your case) to a substantial smaller subset of points, between which linear interpolation is an acceptable approximation of the original array/function. It is a speed - memory - precision tradeoff.

Might help,
Rob

gravelbar:
Also he as advised using "const" for variables, this put them in flash, correct?

In this case, your pro-friend is wrong. const will not place constants/variables into FLASH. They will still consume RAM during the program's execution. Only variables which are explicitly placed into PROGMEM get put into FLASH.

The only thing "const" really does is generate complier errors if your code attempts to modify a "constant."

OK everybody, here's the code.

THANKS for your help! I've put description Exhibition / Gallery to make things neater:

http://arduino.cc/forum/index.php/topic,78866

Should mention programmer X (anonymous pal) wrote some very sweet encoder code; seems to not be enough of that around; it's in there.

Steve

PS. Yes, I know all the "30"'s in hydrographs can be replaced, we eventually want to have other numbers there

Isn't the Arduino compiler smart enough to do that with static consts? Many compilers are.

Not a question of smart enough, think about how you might have a string constant that could be in flash memory, and then passed to Serial.print.

AWOL:
Not a question of smart enough, think about how you might have a string constant that could be in flash memory, and then passed to Serial.print.

Are you referring to the possibility of casting away const-ness?

No, I'm referring to the difficulty of distinguishing entities in completely different memory spaces.

We have tons of flash left over

Progmem is an answer then. You don't really need anything more than one step of your hydro instructions in RAM proper. As I haven't used progmem before, I hacked this together as an experiment:

#include <avr/pgmspace.h>


// One step within a hydrograph.
struct hydrograph_step_t
{
  // The duration of this step in seconds.
  prog_uint16_t seconds;

  // The motor PWM value during this step (0-255).
  prog_uchar pwm;
};

PROGMEM struct hydrograph_step_t Graph1[] = { {30,100},{30,110},{30,255},{0,0}};
PROGMEM struct hydrograph_step_t Graph2[] = { {30,1  },{40,90 },{50,128},{0,0}};

struct HydroGraph_t
{
char *Name;
hydrograph_step_t *Steps;
};

struct HydroGraph_t HydroGraphs[]= {{"Hydro1",Graph1},{"Hydro1",Graph2}};

struct hydrograph_step_t  ThisStep;

void setup()
{
Serial.begin(9600);
Serial.println("Progmem Test");

memcpy_P(&ThisStep,&(HydroGraphs[1].Steps[2]),sizeof(ThisStep));
Serial.print(ThisStep.seconds);
Serial.print (" ");
Serial.println(ThisStep.pwm,DEC);
}

void loop()
{
}

Seems to work

AWOL:
No, I'm referring to the difficulty of distinguishing entities in completely different memory spaces.

By 'eck, I had no idea all that was going on. I've been spoiled by systems with a unified address space.

Embedded is a strange discipline.
You can have von Neumann architecture, with ROM and RAM but in the same address space (old style microprocessors), or Harvard, with different spaces for both (classic microcontroller)

On the PROGMEM page it does say:

Using PROGMEM is also a two-step procedure. After getting the data into Flash memory, it requires special methods (functions), also defined in the pgmspace.h library, to read the data from program memory back into SRAM, so we can do something useful with it.

So far every post I've seen about having problems using PROGMEM did the setup but not the special methods.

Thanks very much wildbill.

Most of the other stuff ya'll are talking about is way above my pay grade.

And please remember, friends, we're sending a few dozen of these out. Code needs to be bullet-proof. All our hardware, too -- it really sucks when users have problems, and so far this is rare.

We're not like Microsoft, etc., we don't like users testing our gear; we try to do that, and let them just teach.

Thanks for you help, we need it, it's all for education, very worthy cause, I support six employees and haven't made a dime myself in five years, but almost there!

I recently implemented some code using a lookup table that should be exactly what you're looking for. You're on the right track as PROGMEM stores the table to your micro's flash instead of SRAM which is exactly what you need to do and is almost certainly what your friend was trying to say. I generated this table just using Excel to create the numeric columns and then a column between them with commas so that it pasted into code perfectly with no hand-writing of this really boring linear table.

To explain what I have below to help you make use of the example I'm doing some LED work where I need to quickly find a hue of the rainbow from a table instead of doing the expensive math formula or worse, depending on your other SRAM requirements, storing the table in SRAM. You'll see my table is essentially just a linear formula but saved me lots of SRAM and in my case, improved speed dramatically instead of math formulas to calculate the values.

First, make sure you include the necessary header file:

#include <avr/pgmspace.h>  //used for PROGMEM

Then declare your table in your .pde:

//192 RGB values (576 bytes) in a linear fading formula in the form of R,G,B,R

prog_uchar tblRainbow[] PROGMEM = {

255	,	0	,	0	,
251	,	4	,	0	,
247	,	8	,	0	,
243	,	12	,	0	,
239	,	16	,	0	,
235	,	20	,	0	,
231	,	24	,	0	,
227	,	28	,	0	,
223	,	32	,	0	,
219	,	36	,	0	,
215	,	40	,	0	,
211	,	44	,	0	,
207	,	48	,	0	,
203	,	52	,	0	,
199	,	56	,	0	,
195	,	60	,	0	,
191	,	64	,	0	,
187	,	68	,	0	,
183	,	72	,	0	,
179	,	76	,	0	,
175	,	80	,	0	,
171	,	84	,	0	,
167	,	88	,	0	,
163	,	92	,	0	,
159	,	96	,	0	,
155	,	100	,	0	,
151	,	104	,	0	,
147	,	108	,	0	,
143	,	112	,	0	,
139	,	116	,	0	,
135	,	120	,	0	,
131	,	124	,	0	,
127	,	128	,	0	,
123	,	132	,	0	,
119	,	136	,	0	,
115	,	140	,	0	,
111	,	144	,	0	,
107	,	148	,	0	,
103	,	152	,	0	,
99	,	156	,	0	,
95	,	160	,	0	,
91	,	164	,	0	,
87	,	168	,	0	,
83	,	172	,	0	,
79	,	176	,	0	,
75	,	180	,	0	,
71	,	184	,	0	,
67	,	188	,	0	,
63	,	192	,	0	,
59	,	196	,	0	,
55	,	200	,	0	,
51	,	204	,	0	,
47	,	208	,	0	,
43	,	212	,	0	,
39	,	216	,	0	,
35	,	220	,	0	,
31	,	224	,	0	,
27	,	228	,	0	,
23	,	232	,	0	,
19	,	236	,	0	,
15	,	240	,	0	,
11	,	244	,	0	,
7	,	248	,	0	,
3	,	252	,	0	,
0	,	255	,	0	,
0	,	251	,	4	,
0	,	247	,	8	,
0	,	243	,	12	,
0	,	239	,	16	,
0	,	235	,	20	,
0	,	231	,	24	,
0	,	227	,	28	,
0	,	223	,	32	,
0	,	219	,	36	,
0	,	215	,	40	,
0	,	211	,	44	,
0	,	207	,	48	,
0	,	203	,	52	,
0	,	199	,	56	,
0	,	195	,	60	,
0	,	191	,	64	,
0	,	187	,	68	,
0	,	183	,	72	,
0	,	179	,	76	,
0	,	175	,	80	,
0	,	171	,	84	,
0	,	167	,	88	,
0	,	163	,	92	,
0	,	159	,	96	,
0	,	155	,	100	,
0	,	151	,	104	,
0	,	147	,	108	,
0	,	143	,	112	,
0	,	139	,	116	,
0	,	135	,	120	,
0	,	131	,	124	,
0	,	127	,	128	,
0	,	123	,	132	,
0	,	119	,	136	,
0	,	115	,	140	,
0	,	111	,	144	,
0	,	107	,	148	,
0	,	103	,	152	,
0	,	99	,	156	,
0	,	95	,	160	,
0	,	91	,	164	,
0	,	87	,	168	,
0	,	83	,	172	,
0	,	79	,	176	,
0	,	75	,	180	,
0	,	71	,	184	,
0	,	67	,	188	,
0	,	63	,	192	,
0	,	59	,	196	,
0	,	55	,	200	,
0	,	51	,	204	,
0	,	47	,	208	,
0	,	43	,	212	,
0	,	39	,	216	,
0	,	35	,	220	,
0	,	31	,	224	,
0	,	27	,	228	,
0	,	23	,	232	,
0	,	19	,	236	,
0	,	15	,	240	,
0	,	11	,	244	,
0	,	7	,	248	,
0	,	3	,	252	,
0	,	0	,	255	,
4	,	0	,	251	,
8	,	0	,	247	,
12	,	0	,	243	,
16	,	0	,	239	,
20	,	0	,	235	,
24	,	0	,	231	,
28	,	0	,	227	,
32	,	0	,	223	,
36	,	0	,	219	,
40	,	0	,	215	,
44	,	0	,	211	,
48	,	0	,	207	,
52	,	0	,	203	,
56	,	0	,	199	,
60	,	0	,	195	,
64	,	0	,	191	,
68	,	0	,	187	,
72	,	0	,	183	,
76	,	0	,	179	,
80	,	0	,	175	,
84	,	0	,	171	,
88	,	0	,	167	,
92	,	0	,	163	,
96	,	0	,	159	,
100	,	0	,	155	,
104	,	0	,	151	,
108	,	0	,	147	,
112	,	0	,	143	,
116	,	0	,	139	,
120	,	0	,	135	,
124	,	0	,	131	,
128	,	0	,	127	,
132	,	0	,	123	,
136	,	0	,	119	,
140	,	0	,	115	,
144	,	0	,	111	,
148	,	0	,	107	,
152	,	0	,	103	,
156	,	0	,	99	,
160	,	0	,	95	,
164	,	0	,	91	,
168	,	0	,	87	,
172	,	0	,	83	,
176	,	0	,	79	,
180	,	0	,	75	,
184	,	0	,	71	,
188	,	0	,	67	,
192	,	0	,	63	,
196	,	0	,	59	,
200	,	0	,	55	,
204	,	0	,	51	,
208	,	0	,	47	,
212	,	0	,	43	,
216	,	0	,	39	,
220	,	0	,	35	,
224	,	0	,	31	,
228	,	0	,	27	,
232	,	0	,	23	,
236	,	0	,	19	,
240	,	0	,	15	,
244	,	0	,	11	,
248	,	0	,	7	,
252	,	0	,	3
};

Then write the small amount of extra code to retrieve the value from the table:

      byte colorNum = 96;
      pgm_read_byte_near(tblRainbow + (colorNum *3)),
      pgm_read_byte_near(tblRainbow + (colorNum *3) +1),
      pgm_read_byte_near(tblRainbow + (colorNum *3) +2)

When I want to pull out an RGB color (for example, color number 96, directly in the middle of the rainbow) I use the following code to pull bytes 288, 289, and 290 from the 576 byte table as shown above.

I hope that gives you a practical example of how to use a lookup table with PROGMEM. If you need more specific help, PM me and I'll look over your exact usage in more detail and tailor a better example/solution for you.

I decided to take a small break and look over your code a little more and have some more specific advice for you.

First, absolutely implement the PROGMEM example I gave above for your hydrograph tables. It will free up 328 bytes, or 20%, of your SRAM right off the top. This might be enough for you right now but if you want to understand a little more or further optimize there's more you can do.

Second, you should also know that each of your literal strings also eats 1 byte of RAM for each character. For example,. the hydrograph names ("8m flashy", "17m triangle", etc) can also be placed in PROGMEM and will free up 67 bytes on their own. You also use 1 byte of SRAM for each character that's only used in a single Serial.print() or lcd.print() including spaces also. For example, Serial.print(" pwmCal ") eats 11 bytes. Hunt down these and either eliminate, shorten, or move to PROGMEM.

Additionally it looks like static uint8_t Qvals[256] can also be moved to PROGMEM as it's read-only and essentially a lookup table as well. Is that correct? If so you're well on your way to cutting your application's SRAM usage by 50% if you can move this as well.

Last thing of mention is that you only want to store things in PROGMEM that are read-only. Writing to flash does have a limited number of times it can be reliably performed (1,000 writes to any given address on the ATMega family I believe) so keep that in mind before you optimize too deeply. Reads are unlimited.