Mega, Due, Sketch 1.5.8 : write/read data to/from flash ?

Hi,
I am urgently looking for a way to write and to read data to/from flash.
It should provide standard C compatibility like sort of stdio.h commands (fput, fputs, fget, fgets, fprintf, fread, fflush...)

How can I do this by my Mega and also by my Due (Sketch 1.5.8.) ?
Stored user data (e.g., like a maze map) must not be deleted if a new updated program version will be uploaded!

So you want to take a static block of memory and treat it as a stream? Or are you going to divide it into separate blocks?

How much space are you looking to use? Would an SD card do the job? Having separate files overcomes the problem of the streamable data YET also allow unrelated data to remain safe.

hi,
its for some arrays of byte, e.g.
byte map[100][100] to store into 1 file
and multple extra data (bytes, ints, floats, strings) into 2 or 3 extra files.
Probably 128-256kB would fit for my Due programs, probably less on the Mega.

Imgine a map-and explore-robot which gathers data about the room/environment and stores them for later runs when being shut off intermediately.
Or imagine a neural net which has to store trained data (neuron inputs, weights, outputs) for later use during training and performing.

I don't want to have to think about blocks or so,
I want to write single formatted data via fprintf and long arrays of structures or 2-dim arrays via fflush or something because of processing speed.
Appending data to files also required.
Reading similar.

Simply just like stdio.h functions, that's what's actually needed

WOW I just did a quick google for 1mbit flash ram and came across THIS

It's been a while since I bought any flash ram but this looks so cheep. Shame it's 3.3v but at the price, I'd be inclined to find a work around.

Do you have the ram chip(s) yet? It shouldn't be too difficult to build a library to do the job.

no, no external memory, I want to use the onboard-flash of course!
the Due got plenty of it:

512 KBytes of Flash memory

Ah this is where your sketch resides. You can access it using progmem.

thank you!
but how exactly?
is there a programming guide?
will it stay on the flash if I re-upload a newer program update or even another program intermediately so that the precious data won't get lost??

You define it in the same way as any other variable but you use progmem. See if this helps

Whenever you upload a sketch it will overwrite your current sketch so, yes, if your new sketch doesn't also include it then you'll loose it.

You could, perhaps include your progmem definitions in a separate .cpp file though to save you having to add it in to every sketch.

the map or different additional data will be computed (explored) during runtime by the program, so there is nothing I know about at compile-time.

The program learns it, stores it to files and must be able to reload it at any time even after a reupload of another program.

It's exceedingly difficult to write to program space during run time. Not quite impossible but quite unpractical. (you'd need to fiddle around with the boot loader code). So that flash ram chip I linked to earlier is looking more appealing. A 5v version would be more helpful though. An alternative would be an SD card. They're very simple to implement (albeit a tad cumbersome to access).

it's no problem, I'm always doing it on my NXT.
ok, as the NXT program is running on a VM on the flash, too, the program execution is stopped during r/w file access, but that's just a fraction of a second.

But if difficult or not -
I urgently need to r/w non-volatile data during runtime, possibly by stdio.h commands, and also after re-upload of a new program, that's what the topic is about.

So not possible for the built-in flash?
that would be really a crap!

So how to r/w non-volatile data during runtime on SD by stdio.h commands instead?
How to wire and how to install a SD card by ICSP/SPI-Headers?
Which SD card SPI board is cheap and works safely with it's C libs for Sketch (Mega + Due compiler, 5V and 3.3V !)?