vars patch saving/memory/storage feasiblity

Using Arduino Project Guidance
1

hi everyone

i have a project scoping question or partly a ram/memory question

I have actually developed the project and its almost done, just putting the hardware together now.....Its a sequencer and is based on the Mega 2560 ......and as I have 7 pins left it got me thinking......... what can I do with these??

The only thing I think I may like to use them for i some sort of patch memory function

Overall I have about 180 declared variables (mostly int) and the code seems to work fine altho I need to tidy it up a hell of a lot!!

So for each 'patch' id like to save there's 56 variables i want to be able to store or recall in an instant and i thought of having 2 banks of 12 patches

2x12x56 is over a thousand variables!!

I've looked up my memory/staorage options:

Flash memory / PROGMEM, it seems you can only save to this once per 'session' so great for loading loads of pre-determined data into arrays for lookup but no good for on the fly data saving no?

SRAM is volatile

EPROM is slow? I want to save/recall while its running, plus if I wrote a thousand variables to it a hundred time each time I use it, it could wear out i think?

firstly am I correct with these findings/assumptions?

secondly would it be feasible given the amount of overall variables I'm using in SRAM, that I could store another 1000 here in 'patch' arrays without running out of space while its running, then just 'dumping' and 'loading' from eprom on 'startup' and 'shutdown'

Am I on the right track? anyone done anything similar?

any ideas on the limits of the Mega in layman's terms, i.e how many ints can you work with in one go?

any help or pointers appreciated

cheers

2

Laktica:
2x12x56 is over a thousand variables!!

That's not strictly speaking true, as I wouldn't expect you'd want to have all the patches loaded simultaneously. It is certainly 1,344 values to store, but that's an important distinction because if you offload them to PROGMEM or EEPROM, or SD card or any of these options you are looking into, there is no need to have them stored in variables. Far more likely you'd only load one patch into a single set of variables when required. Hope that makes sense.

Laktica:
Flash memory / PROGMEM, it seems you can only save to this once per 'session' so great for loading loads of pre-determined data into arrays for lookup but no good for on the fly data saving no?

That's a pretty reasonable summary. It's typically used for storing static data, and remembering it goes away between runs, you'd need to store it again each time you start...which means you'd not be able to change the patches permanently as the patches would return to whatever was in your code when you start over.

Laktica:
SRAM is volatile

Indeed it is. And you also have the least of it of the different memory types available. So it's not an option for saving your patches, unless you don't mind them vanishing when the power turns off, or the reset button is pressed.

Laktica:
EPROM is slow? I want to save/recall while its running, plus if I wrote a thousand variables to it a hundred time each time I use it, it could wear out i think?

EEPROM is the slowest of the memory types onboard the processor, however according to the reference page an EEPROM write takes 3.3 ms to complete so if you're going to save a single patch of 56 integers (from your description) that's just over 1/3 sec to write the 112 bytes. Depending on the frequency you'd be expecting patches to be written, that could be quite acceptable. In this application, the 100,000 erase/write cycle maximum is also probably not going to be tested from what you've typed. So I don't think you need worry about wearing it out - saving these patches is very much like saving settings and that's a perfect use for the EEPROM onboard. Remember too, you can add external EEPROM chips if you want more too...

Laktica:
secondly would it be feasible given the amount of overall variables I'm using in SRAM, that I could store another 1000 here in 'patch' arrays without running out of space while its running, then just 'dumping' and 'loading' from eprom on 'startup' and 'shutdown'

You could, but with only 8kB RAM to play with, it's probably a better idea to fetch them when you need them only, as I presume you're not going to need all of them at one time.

Laktica:
any ideas on the limits of the Mega in layman's terms, i.e how many ints can you work with in one go?

Well here's where it's not so straightforward. The size of RAM required to run any program changes during the run of the code. This is impacted by the flow of the program, what functions are called, and variables being created within functions and then destroyed when the functions end. The best I can advise here is to avoid putting big data structures into your code in global scope as it is the most scarce resource you have to play with.

Hope this helps,
Geoff

3

the most helpful reply I've had in this forum, I really appreciate your response, THANKS!

strykeroz:
That's not strictly speaking true, as I wouldn't expect you'd want to have all the patches loaded simultaneously. It is certainly 1,344 values to store, but that's an important distinction because if you offload them to PROGMEM or EEPROM, or SD card or any of these options you are looking into, there is no need to have them stored in variables. Far more likely you'd only load one patch into a single set of variables when required. Hope that makes sense.

You are spot on about the way I want it to work. From now on I'll make a distinction between variables and values, might help my questioning!

Thanks for confirming my memory findings, looks like PROGMEM is defo not the way to go!

strykeroz:

Laktica:
any ideas on the limits of the Mega in layman's terms, i.e how many ints can you work with in one go?

Well here's where it's not so straightforward. The size of RAM required to run any program changes during the run of the code. This is impacted by the flow of the program, what functions are called, and variables being created within functions and then destroyed when the functions end. The best I can advise here is to avoid putting big data structures into your code in global scope as it is the most scarce resource you have to play with.

Sorry this way a bit of an afterthought vague question! I realize its not particularly answerable now!

The reason I was asking about this and the " 'dumping' and 'loading' from eprom<>SRAM on 'startup' and 'shutdown' " idea is because, while the sequencer is running I don't want an glitches in ITS timing, if vales take a few seconds to save/load that's fine as long as it doesn't affect the overall timing counter. So maybe temp vars in SRAM until the sequencer is stopped is the only solution to this, I'll investigate.

For EPROM you mentioned "Depending on the frequency you'd be expecting patches to be written, that could be quite acceptable." but I don't see why, Is it because during the 1/3 sec the sketch can't do anything else (i.e. the sequencer timing will stutter) or are you relating to the actual data incoming/outgoing buffering issues with the EPROM? If its the latter I don't see it being a problem as I'll only want to save patches now and again, if its the former I will have to investigate some sort of 'SRAM buffer' as I mentioned above.

thanks again.

4

Hi again,

Laktica:
the most helpful reply I've had in this forum, I really appreciate your response, THANKS!

You're welcome. thanx :slight_smile:

Laktica:
For EPROM you mentioned "Depending on the frequency you'd be expecting patches to be written, that could be quite acceptable." but I don't see why, Is it because during the 1/3 sec the sketch can't do anything else (i.e. the sequencer timing will stutter) or are you relating to the actual data incoming/outgoing buffering issues with the EPROM? If its the latter I don't see it being a problem as I'll only want to save patches now and again, if its the former I will have to investigate some sort of 'SRAM buffer' as I mentioned above.

That depends very much on the implementation. You could of course write it a byte at a time and keep doing other things in between. This is the time cost of writing only, reading is faster but I didn't find a spec on that.

And remembering if you use PROGMEM you won't be able to have a user modify the patch and save what was changed, they'll effectively be presets only. PROGMEM is a way of storing data that you know the values of when you write your sketch, like constant strings or large lookup tables, without taking up SRAM until you need it. EEPROM is read/write during the runtime of your sketch so good for saving settings, high scores, logged sensor data, and other things that change during the run that your sketch needs to keep track of next time it's powered up.

Depends entirely on what you want to do. Lots of options. Cheers !
Geoff