I know, from what I have read, that the stock Arduino device comes with a fixed amount of memory that can be used for running sketches, ranging from about 4k, to 256k or so, depending on the Atmel chip used.
I also know that by using a SD shield, or an Ethernet shield that has a built-in SD card reader, I can use the SD chip for data-logging and such.
Question #1:
Has anyone tried to use the SD card's memory for extended data storage, used programmatically? For example: I want to use an Arduino with an Ethernet shield, (containing an SD card reader), as a web server and I want to put multiple web-pages on the, (much larger!), SD card rather than main memory?
Question #2:
Is it possible to use the external memory on the SD card as additional storage for larger sketches than main memory will hold? Can I execute out of the external memory, or is there a paging scheme that would work?
If this hasn't been done, than this is an area ripe for research, but if it HAS been done - at least in part - I'd really rather not re-invent the wheel again if I don't have to.
jharris1993:
Question #1:
Has anyone tried to use the SD card's memory for extended data storage, used programmatically? For example: I want to use an Arduino with an Ethernet shield, (containing an SD card reader), as a web server and I want to put multiple web-pages on the, (much larger!), SD card rather than main memory?
I have.
Isn't that what SD cards are for...?
jharris1993:
Question #2:
Is it possible to use the external memory on the SD card as additional storage for larger sketches than main memory will hold? Can I execute out of the external memory, or is there a paging scheme that would work?
If this hasn't been done, than this is an area ripe for research, but if it HAS been done - at least in part - I'd really rather not re-invent the wheel again if I don't have to.
ATmega Chips can write to their internal flash, yes. The Arduino bootloader does it all the time. You could load sketches off the card with a similar scheme.
jharris1993:
Question #1:
Has anyone tried to use the SD card's memory for extended data storage, used programmatically? For example: I want to use an Arduino with an Ethernet shield, (containing an SD card reader), as a web server and I want to put multiple web-pages on the, (much larger!), SD card rather than main memory?
I have.
Isn't that what SD cards are for...?
Oh so true. . . .
However, if I had $20 for every time there was a silly restriction on how some memory media can be used, I'd be a gazillionare today. And it would not surprise me if there was some restriction within the Atmel chip that allows the bootloader to load from USB, but won't allow similar activity to happen at run-time.
Sorry to have asked such a silly question, and I do thank you for the time you took to answer it.
holmes4:
In reply to question 2 yes you can it called overlay programming. Its a very old trick from the 1950's/1960's but ..... well look it up.
Mark
Mark,
You are absolutely correct and it has gone under a variety of names over the years. In fact, modern operating systems use a virtual memory scheme that - at the kernel level - is rather similar.
I was thinking about trying something like that, but wanted to ask first BEFORE banging my head against the floor trying to do something that everyone else knew could not be done.
You may also like to look at code relocation where addresses are resolved at load time or using only addressing via registers for both jumps and load/store op's. But to be realistic you need to asm only! and it's very hard work to write such programs. Better to get a bigger CPU!.
If you wanted to approach this idea from another angle, one way to do what you want to do would be to write an interpreter that would fit within the limits of the SRAM and Flash; there is likely more than enough space available for that.
It would no longer be an "Arduino" (that is, you couldn't develop using the normal Arduino tools) but that may not be that big of an issue. You'd have to make the interpreter very lean, and it would likely only run on a single ATMega processor. The biggest issue would likely be how to keep the SD card reader code tight and lightweight. It would take a lot of work, but it should be possible. It wouldn't be as fast as a regular Arduino, given the interpretation overhead, but it would work well for most tasks, I think. It might even be easier to develop for.
That's basically what the Parallax Basic Stamp 2 was - a PIC16F57 microcontroller with an external 24LC16B EEPROM. The PIC ran at 20 MHz, with 2K (12 bit) words of flash, 72 (8 bit) bytes of SRAM; the EEPROM was a 16K I2C device. There shouldn't be any reason why you couldn't do the same with the 328 and a similar EEPROM or an SD card.
Hate to rain on your parade, but I don't think this is possible (I mean putting sketch instructions on an SD card and having the arduino execute them). At the very least, it's going to be insanely complicated.
First of all, the VM paging and program overlay methods you mention are for computers using a Von Neumann architecture. These computers have the instructions and data in the same memory space. Therefore they can read instructions into memory and then set the program counter to that memory location to start executing them.
The arduino is a Harvard architecture in which the instructions and data are in separate memory spaces. It cannot execute an instruction stored in the RAM. So you can't read a compiled sketch from an SD card into RAM and then start executing it.
So how does the arduino put a sketch into program memory? The bootloader sucks up a block of instructions from the serial port into RAM and then uses some special instructions to stuff it into program memory. There are lots of limitations: you have to do a block (page) at a time, the timing is critical, the source and dest addresses have to be aligned correctly, etc. See the data sheet for more information. The bootloader and the program are segregated so that you can't trash your bootloader by downloading a sketch.
It may be possible to write a sketch that reads instructions from an SD card into RAM and then loads them into the program memory. I doubt it would be easy, though. You can't over-write any part of the code that is doing the reading including the SD interface. Since you can't be sure where the sketch is in program memory, you'd probably have to write the sketch in assembler, like the boot loader.
Plus there's the problem of knowing WHEN to read in a page of instructions. There are no page faults on an arduino. By the time you write the code (probably in assembly) for the SD interface, pushing into progmem, and doing the code paging, you've probably eaten up most of your memory anyway. It would be much simpler to just use an arduino with more memory.
Then there is the problem of writing the code that goes on the SD card. You wouldn't be able to use regular arduino sketches, that's for sure.
It would be an enormous coup to get it to work in some sort of reasonably usable fashion. I think it would be really slow, though, and the code would be fragile. Probably hard to use on different arduino boards, too.
Dealing with the memory limitations (both RAM and progmem) of the arduino is just part of the experience. If you want more than that, you need to look at a different kind of microcontroller. There are atmel chips that support external RAM. And things like Raspberry Pi that have full VM support.
PicAxes actually have commands that can do this, but then they use bytecodes passed to the internal BASIC interpreter. Of course, PicAxes or so limited in memory, that they have to be able to do this. 8^)
jharris1993:
I know, from what I have read, that the stock Arduino device comes with a fixed amount of memory that can be used for running sketches, ranging from about 4k, to 256k or so, depending on the Atmel chip used.
I also know that by using a SD shield, or an Ethernet shield that has a built-in SD card reader, I can use the SD chip for data-logging and such.
Question #1:
Has anyone tried to use the SD card's memory for extended data storage, used programmatically? For example: I want to use an Arduino with an Ethernet shield, (containing an SD card reader), as a web server and I want to put multiple web-pages on the, (much larger!), SD card rather than main memory?
Question #2:
Is it possible to use the external memory on the SD card as additional storage for larger sketches than main memory will hold? Can I execute out of the external memory, or is there a paging scheme that would work?
If this hasn't been done, than this is an area ripe for research, but if it HAS been done - at least in part - I'd really rather not re-invent the wheel again if I don't have to.
Thanks!
1 yes it's pretty trivial to do. It does use a pile of sram as the sd card is fat formatted thus block oriented so your reading 512b at a clip. Please be specific I'm assuming your talking about flash as main memory though that's what most would call sram. If not take a look at progmem PROGMEM - Arduino Reference before thinking you need to deal with sdcards.
2 Yes but it's a great big PITA far easier to get a chip with more flash like the 1284 than to fiddle around with this. This method is also very SLOW and wastes sram to do so. If you think you need more flash than a 1284 provides consider if an 8 bit micro is the right processor type for your application.
That sounds like an enormous sketch. Just out of curiosity, what's it doing that needs all that code space? If it's all genuinely needed, it suggests to me that the Arduino is probably not the right platform for whatever it is you're trying to do.
PeterH:
That sounds like an enormous sketch. Just out of curiosity, what's it doing that needs all that code space? If it's all genuinely needed, it suggests to me that the Arduino is probably not the right platform for whatever it is you're trying to do.