I want to be able to edit an Arduino program on the fly without pc, however I don't want to rely on an interpreter on the arduino because I want maximum speed and maximum program space.
I realize I assume there are no interpreters on the Arduino which are approx. as fast as native code. If this is wrong please correct me.
To make above possible I was thinking about the following setup:
Two arduino's sharing same input (keyboard/terminal in) and same output (lcd-screen/terminal out)
The primary arduino handles program editing
If finished the primary arduino 'compiles' and uploads it to the secondary arduino
CrossRoads:
So a small PC running compiler & avrdude. Could a Rasp Pi do that?
There are people who have the Arduino Development Environment hosted on a Raspberry Pi.
The thing I don't get about this sort of project, from a practical standpoint, is that if one needs a keyboard and display there seems to be little reason not to just use some flavor of laptop.
Why you are not using a computer is not explained. If it is budget, you can get a cheap win 10 netbook for $150, like asus. If it is size, the laptop is just 11.6'.
NardJ:
I want to be able to edit an Arduino program on the fly without pc, however I don't want to rely on an interpreter on the arduino because I want maximum speed and maximum program space.
I realize I assume there are no interpreters on the Arduino which are approx. as fast as native code. If this is wrong please correct me.
To make above possible I was thinking about the following setup:
Two arduino's sharing same input (keyboard/terminal in) and same output (lcd-screen/terminal out)
The primary arduino handles program editing
If finished the primary arduino 'compiles' and uploads it to the secondary arduino
Secondary arduino runs program
Would this be the right way to go?
You will probably have to write your own special compiler for your own special programming language.
Normally program = code + data, and we modify the data which to you might be an interpreter of some kind but I'd put that up against a lot of code I've seen for speed and size. Data can be streamed in or even generated while changing the code requires run time to stop.
I want to change code on an Arduino without having to use a pc. As I understand the Harvard architecture makes this impossible using the (same) Arduino itself and I need an external device to write the new changed code. Why I don't want to use a pc, is not cost. I just want a standalone full Arduino solution more as a challenge than anything else.
I probably shouldn't have used the word 'compile'. Write some very low level code (machine code/assembly) on the first Arduino is enough in which case there wouldn't be much compiling, just uploading it to and running it on the second.
If you already have compiled hex files to load then you could use Nick Gammon's hex loader:
If you want a challenge then find out how the compiler we have now works. Then if you want to roll your own you will be knowing ahead what needs to be done.
I have not studied the subject but I suspect that Arduino A could be programmed to change the code in Arduino B. After all, that is what happens when you use one Arduino to upload the bootloader on another Arduino.
Whether you can do anything useful is another matter. In the case of the bootloader the code being uploaded will have been created on a PC.
NardJ:
I want to change code on an Arduino without having to use a pc. As I understand the Harvard architecture makes this impossible using the (same) Arduino itself and I need an external device to write the new changed code. Why I don't want to use a pc, is not cost. I just want a standalone full Arduino solution more as a challenge than anything else.
The atmega328 has the capability to write to program memory via the "Store Program Memory" (SPM) instruction. In Arduino, the bootloader receives a representation of the compiled code via USB and burns it to program Flash memory by this mechanism. In Arduino SPM is a "privileged instruction" in that you can not use it from the user memory space, so I don't think you can write a monitor in user space called by the stock Arduino bootloader. It should be possible to replace the bootloader with a monitor of sorts that interfaces with your I/O (say a hex keypad and a multi-digit 7-segment display) and have that monitor burn the code you enter to user program flash memory and jump to it. That would give you a single atmega328 solution to your problem as I understand it.
The point of Arduino is to bypass this sort of very low level interaction, so it's well outside of the Arduino paradigm, but it's how everyone did it back in the early days of hobbyist microprocessors. It might make for an educational exercise, but there are good reasons why no one does it that way anymore.
I now start to understand you. You want, as a challenge, to change code on an arduino without using a PC, but the change of code is from one compiled code to another compiled code. So how do you trigger the change, a button push? As MrMark explained, an arduino CAN change its own code. It's not strictly-speaking a Harvard architecture that prevents a processor from overwriting its own code space. When the processor is running bootloader, it CAN write its own FLASH, up to the end of read-while-write section. The bootloader resides in the non-read-while-write section so it won't overwrite itself.
If you're starting with a baseline Arduino like Uno, the only self-contained language that would fit in that little space would be Forth. Forth will compile to something, that something might be indirect-threaded code, direct-threaded code, subroutine-threaded code or plain old native code and there are advantages and disadvantages to each. On a conventional CPU, subroutine-threaded code has the advantage that the compiler can be incrementally migrated to native code, but on an 8-bit microcontroller there is a limit to the depth of the return stack (I'm familiar with Microchip MCU's, but not Atmel, I'm assuming they're similar). If you're really concerned about program space and speed, I would go to one of the 32-bit Arduinos like Zero or Due, aside from their greater capabilities, they have much nicer assembly languages to work with. As program space grows, you have more options, if you don't like Forth, you could write a small Basic compiler. Of course, that is all a lot of work. Interpreted code on a Due might well be fast enough for your needs, an existing interpreter written in C would save a lot of time.
Thx, the SPM instruction seems what I need, too bad it cannot be called from a program in user space memory .
Just to be sure, but is there any workaround known to call this instruction while running a program. In other words, is it possible to write an monitor/editor which (partially) resides in user space?
If I would be able to write an editor in the memory space where now the bootloader resides, would it be possible to switch between running the editor and the edited program without resetting the Arduino?
There's not much point in having an editor in user space, since you can't run anything in user space. If you're implementing an interpreter, you can keep interpreter tokens in user space and a program in program space will interpret them. This need not be terribly slow, if your tokens are addresses in program space, your interpreter can be very simple, it increments a pointer into your token list and calls the subroutine in program space which is the run-time for your interpreter token. But in this scheme you're not generating native code, you're generating interpreter tokens. But you also need to think about what you need when you're talking about a "program". I assume you are talking about editing the source code, that never needs to go into program memory. If you are compiling to native code, that will go into program memory. However, if you're going to retain the ability to edit that code, you need to somehow associate the source code with the object code. Unless you want to recompile the entire program every time you make a change. When you parse the source code, you could associate every token (here I mean a unit of meaning in the source language, like a reserved word or variable name) with a link which tells where the native code associated with that token is located. Another problem occurs if your edited code generates more object code than the original version and it's not the last code in your program. Then you have to relocate all the following code.
All of this should be telling you why they didn't put the compiler on the MCU in the first place. There isn't enough program space to implement all this and if you were some kind of super-genius who could prove me wrong, there still wouldn't be any room for a user program. Your goal of saving memory space would be completely overwhelmed by the program space used for your editor and compiler. Again, this applies to the 8-bit versions. If you're bound and determined to do this, get one of the 32-bit Arduinos. You're going to be putting thousands of dollars of your time into implementing this, the cost of one of the more powerful boards will be meaningless.
And of course the easiest solution has already been mentioned, get a Raspberry Pi or Beaglebone and use the development system that comes with it.
NardJ:
Thx, the SPM instruction seems what I need, too bad it cannot be called from a program in user space memory .
Just to be sure, but is there any workaround known to call this instruction while running a program. In other words, is it possible to write an monitor/editor which (partially) resides in user space?
You want to look at section 27 of the Atmeg328 data sheet. The microcontroller has modes which allow writing to user space EEProm. As I understand it (which isn't authoritative), this isn't allowed in the "Arduino" environment. That is, it can be done with Arduino hardware (e.g. Uno) but not entirely within the Arduino software development and flashing environment.
Looking around a bit, I see that there is at least one Forth project that apparently replaces the bootloader with Forth which serves as self-contained monitor, compiler, and interpreter and which allows on-chip storage of compiled code input via the monitor. There also appear to be some Basic projects which have serial monitor and interpreter, but it's not clear to me that any of these can store user code in non-volatile memory on chip. It seems like the Basic guys are using SDcards or serial EEPROMs for program storage.
There is a reason you should not write to FLASH your code is running from. IF you want to write to FLASH, you can only do it inside the bootloader memory block, designated by bootloader size BOOTSZ bits in the fuse. You could, if you would, develop a custom bootloader for the first arduino to load a new sketch supplied by the second arduino. I've only done a little bit of bootloader development. It's doable but you need more space than 512 bytes currently used by the optiboot, maybe 1KB or 2KB depending on how complex your logic is. Say you just want first arduino to receive a new sketch whenever it gets reset, then the second arduino just has to run a modified version of Nick's hex uploader with an SD card.
Tell us something more specific, like, "this is what I want to do". If pre-compiled code is all you want to send to arduino by another, then read Nick's hex uploader. If you want to dynamically change parameter values of a source code and compile it into hex then upload, then arduino can't do it. You need raspberry pi.
