May I please get your guidance how to approach problems when the sketch doesn't fit onto a single device? Assume that I've preselected the largest device available in its class, but that the program needs 150% -250% of the single device capacity.
Assume also that F strings and PROGMEM are already considered also thanks.
Are there any guides which may help me understand correct coding techniques and alternative circuit designs which effectively extend the available flash space? My queries are general in this case, however, I can see scenarios in the near future where a single microcontroller flash is insufficient to store the program, and SRAM will need to carefully allocate and release to support a greater volume of variables.
ian_west:
May I please get your guidance how to approach problems when the sketch doesn't fit onto a single device? Assume that I've preselected the largest device available in its class, but that the program needs 150% -250% of the single device capacity.
I doubt that. If that is the case it must be something very specific and can't be solved with general advice.
ian_west:
Are there any guides which may help me understand correct coding techniques and alternative circuit designs which effectively extend the available flash space?
The flash is very tight with the cpu, it takes 3 cycles to fetch a byte from flash.
What's more, the Instruction Pointer points into flash, not RAM on an MCU.
There is no extending that.
You can hang flash chips for data storage on an Arduino, SPI bus would be fastest, and the same bus could have SD cards as well. Wiring all the select pins, a demux chip would save Arduino pins.
I have looked at ATmega1284 as a data server since it has 16K RAM == lots of 512B buffers.
The Mega2560 has 4 high speed serial ports and in ways looks a lot like a Transputer chip only smaller.
I have a somewhat figured out but not coded way to split tasks across controllers by passing serial messages around a ring of chips. Each TX gets wired to the next-in-ring RX.
Each chip has a list of "command words". If serial reads one, the chip runs a matching function -- I do have that working btw.
Messages start with chip #. If a different chip originated a message, the chip copies the message onto TX as it arrives.. otherwise no copy.
I write tasking code on Arduino. I wrote a lot of code for money in the 80's and 90's. I worked hard the whole time on reducing necessary code. The less there is, the less I have to debug. I prefer to push details into data and write little systems.
Wait for the next generation of micro, with more memory.
You probably don't need to "wait."
I mean, you have the MEGA topping out at 256k/8k.
And then the SAMD21 boards, with 256k/32k, better code efficiency for math, and options with an SD card (gigabytes!) and/or SPI flash (~4MB)
And the SAMD51 boards with 1MB flash and 256k RAM. and SD/SPI flash.
And the ESP boards with 4MB flash (though "it's complicated.")
And the Raspberry Pi with up to 8GB of RAM (and a big SD card, USB drives, etc) A different class of machine, but ... not that much bigger or more expensive.
I'd guess that the biggest thing you should investigate is moving data to external storage. You can attach gigabytes of flash memory to an ATmega328, but accessing it efficiently may take some work.
ian_west:
May I please get your guidance how to approach problems when the sketch doesn't fit onto a single device?
My first thought is that you are thinking of using the wrong device for your project. Why would you consider using a microcontroller for a project that is unlikely to fit on it?
And my question is not about how many bytes your project needs but about the whole philosophy of how the project should be implemented.
For example it would probably be much easier to implement a large project in Python on a RaspberryPi (or a cheap laptop which comes with an in-built uninterruptible power supply) and maybe offload a few tasks to one or more subsidiary Arduinos.
And if you need more advice then you do need to describe the specific project you are contemplating.
More capable hardware is an obvious choice. There's almost certainly some saving to be made somewhere in your code, but if you need twice as much space as you have, shrinking it to fit will be quite a task.
Beware that the Pi and its ilk may not fit your needs. Because of the multi-tasking O/S, it cannot be relied on for anything that needs tight timing e.g. Adafruit's Neopixel devices.
But perhaps your algorithm or data structures can be improved. Maybe some research into alternative methods would be fruitful. I would guess not for the reduction you need but who knows, perhaps you chose unwisely.
Maybe you can offload some of the work: provide web services on a PC to do the heavy lifting and leave the Arduino doing I/O only. Similarly, IIRC, the Pi was envisaged early on as using a microcontroller as its interface to hardware - that's why its GPIO current limits are so feeble. You could take that path.
