Dynamically loading code

Hello all,

I'm trying to design a system that works with a number of arduinos and other devices (ESP6288) where each device picks up a message from a queue (using MQTT) and then executes a calculation with the data from the message.

At some points I would like to change the calculation and that "formula" (method in code) needs to be put on each device. Of course this would be trivial if it were possible to put the code in a message (using a separate queue) and then putting that method in the code that the device can execute. This will prevent me from needing to load a new sketch in each device.

Is there any known way to achieve this? Perhaps I'm not really good at looking for this on this forum but I did not find any past posts to help me out ...

thx for any info.

Need more info.

How different are all the various "formulas" you'll need to use? Are all the possible choices known ahead of time? You could just code them all and select the proper one at run-time. Or, maybe they're all very similar and only differ by run-time parameters or coefficients?

Show some examples.

Store the formula in EEPROM, and write a function that works as an interpreter to process the formula (basically write a calculator sketch).

Dynamically loading code is not a usual method used for microcontrollers. Most microcontrollers are never changed once they are deployed, they work on tasks that could potentially harm people if things go wrong.

Additionally, dynamic code needs address translation (MMU - Memory Management Unit). Microcontroller processor do not have this capability, because it costs silicon area and makes processing nondeterministic.

https://en.wikipedia.org/wiki/Memory_management_unit

To update functionality microcontroller applications, use an update method that usually requires the user to initiate the update and users are warned that they are not allowed to disable power. This allows a bootloader to remove the entire program and replace it with an updated version. This will come more into fashion now that wireless communication is used for things that are connected to the internet and updates will be required more often to close security holes.

Another issue is, that many microcontrollers use internal flash memory and it is not possible to erase a small part of the memory, but you have to erase large blocks or even the entire flash memory. This again is because of cost and the architecture of the flash memory.

https://en.wikipedia.org/wiki/Flash_memory

Thank you all for the ideas.

Perhaps a little bit more information, I'm just thinking about a system so I'm not building it yet.

In the most simple form, let's assume I'm building a chess engine. In 'simple' AI, a search algorithm will look for an optimal solution (e.g. depth first search, alpha beta with cut off ...). in order to determine 'optimal' it might make use of a formula that calculates the 'value' of a given position. (e.g. the input is a FEN notation, the output an integer representing the value of the position).

This would require an AI to calculate the value of every possible position in the search tree.

I could just write a formula to do so and run the whole program on my PC. I think it is more interesting to build a distributed system that calculates these values before hand (let's say the most common positions) and does this using a cheap network of agents (i.e. arduino, ESP8266 ...).

Knowing myself, I'm sure that I will start with a really simple formula but will want to update it to ensure it plays strong against human players. Each update would of course need to recalculate all positions (well a subset determined by space on my HD).

Simple example, the very first calculation used is often just scoring all pieces (pawn = 1, knight = 2, bishop = 3). After a while you might notice that the value of a knight might be 2.5 of the value of a bishop = 3 if you have a pair but 2.5 if not ... This new formula now needs to be used by the agents (arduino).

david_2018: The idea of using an EEPROM seems like an elegant solution, I could potentially hook up all arduino's to the same EEPROM so I would need only one.

This sparks a new idea, I could build a PCB with a load of atmega chips, using SPI or something similar to communicate with a master (that gets all the positions and the formula from MQTT) and build a sort of parallel specialist computer. I just don't have the skills or knowledge yet to do so :-)

gfvalvo: The suggestion of parameterising the formula will work in the beginning until you decide that the value of a knight differs from the begin and middle / end game. Of course I could try to write all formulas beforehand and then select the correct one with a parameter but that would not allow me to add a new formula without uploading new sketches to all agents. It will however be good enough to start with.

Klaus_K, you are correct, I come from a PC/GUI programming background and the design patterns can not just be transferred to the microcontroller world. I will need to further inform myself on how microcontrollers are networked and how they communicate with each other (and generally how they work). Thank you for the information.

This is rather out of the mainstream use of microcontrollers, especially if you want to change their programming automatically.

If you want to experiment with distributed systems, I'd suggest that a bunch of wifi enabled Pi zeros would be more suitable.

An arduino seems a very poor choice for something that requires rapid calculations in massive amounts. That is sonething you see more often using video card processors, although a large array of raspberry pi's might work well.

Both wildbill and david_2018 are of course correct but I already build a supercomputer using Raspberry Pi's (and some ESPs and arduinos) so that dose not present an interesting project for me personally.

I should have clarified, this is an interesting problem for me, I'm looking for the most performant / cheap / relevant solution. I think building something like I described using microcontrollers instead of SoC or even simply cloud computing will teach me a lot of about microcontrollers.

I think, when you want to learn about microcontrollers you should investigate the peripheral systems of the different products available more than the processor. Many new microcontrollers have implemented the ARM Cortex-M processors because the processor is not the differentiating factor for 32 bit. You need an easy to program platform where software can be reused. There are computational differences, but they are not as different as the peripheral sets.

What differentiates all the 32 bit microcontroller are the capabilities to measure and create electrical signals with as little processor intervention as possible. Developers can now choose microcontrollers that are targeting very specific applications.

  • ultra low power to be able to run on a small battery for sensors e.g. water/gas meters
  • special PWM and ADC for motor control, power conversion
  • radios for wireless applications
  • functional safety support
  • different specialist wired communications USB, CAN, LIN, Ethernet, industrial buses
  • security
  • ...

There is no one microcontroller that does it all, because that device would not be cost competitive.