robtillaart:
Don't get me wrong, I like the concept of storing sketches in the device, it is an inspiring problem and got me thinking. It has several advantages (especially finding the right code as you pointed out) no doubt about that, but it is not a final solution.
This problem is called deployment management. [very recognizable]
I've also heard it variously called Release Control, Release Engineering, Product Engineering, and other various things over the years. I have yet to find a company that has completely nailed it.
robtillaart:
As a developer I need a solution that I can trust, it should work every time I want to. Storing sketches in the Arduino is not allways possible (due to size) and therefor I cannot rely on it.
I find this an interesting view, because the '328 (or any processor) has a finite set of resources that your program could eventually outgrow. There are always "grey area" resources like circular logs in RAM or extra debug ports that you jettison along the way as resources get tight. Source-in-flash is just another such thing that you could use until the program outgrows it. You then decide that (1) it is OK to stop using this feature, or (2) it is valuable to your workflow and you find a bigger chip.
Embedded developers are always making tradeoffs about how to deploy the resources of the platform. Would you give up on circular log buffers or debugging ports because some designs don't have enough resources to afford them? Of course not - they are tools that you deploy when appropriate. Source-in-flash is just another tool to be deployed when appropriate. That doesn't make it unreliable, it just makes it another decision in the tradeoff calculations.
I do get your point that just storing a reference to the source is orders of magnitude less expensive than storing the source, drastically altering the tradeoff decision.
robtillaart:
So if one wants to spend energy in solving deployment management for Arduino, we should think of a way that is:
- transparant for the programmer - (don't do things that can be automated --> KISS)
- configurable (switch on/off etc)
- works for all deployments
- robust, reliable
- and so on.
Storing a sketch in an Arduino does not work allways, as it fails on a crucial point imho SIZE. That doesn't mean it has no value, on contrary it can be very usefull as you pointed out, I am just stating it isn't reliable enough. Storing a reference to the source (etc) takes 16 bytes (UUID) which is 0.05% of the available memory and independant of sketch size. And yes there will be applications that don't even have these 16 bytes free. A real final solution should even work for this case. That means that the reference to the code should be stored in the Arduino but at the same time can't be stored in the Arduino. This is a typical TRIZ contradiction.
Solving that contradiction => the binary code itself is the reference (mmm 32K keys, no good...)
making 32K keys more practical: after uploading a sketch, AVRdude reads the complete memory back and makes a SHA1 hash to be used as reference to the sourcecode. So if one arduino comes back through the mail one can read its memory back, do the SHA1 hash and one has the reference to the sources.
That said, this reference will not be the only way to access the sources, full text indexing of all your sketches is very well possible these days, so such things need to be in the final solution too.
The complexity to realize SketchWithin, SHA1 and UUID is comparable. The differences between the SHA1 and UUID versions are
- SHA1 will generate a new code for every source iteration, where with the UUID this is optional
- UUID uses (at least) 16 bytes, SHA1 uses 0 bytes of Arduino memory
- SHA1 will detect image tampering, UUID will not (key lost??)
- UUID will probably be faster than SHA1.
My final choice would be using UUID, and the SHA1 at release moments. The cases I need the last 16 bytes I should really consider an new larger platform.
EPILOGUE:
In short storing a sketch in the Arduino is usefull in many cases as you pointed out. However it won't solve the "what version of code have I deployed problem" in all cases. The SHA1 and UUID solutions will perform better especially for large sketches. My choice would be using UUID all the time, and the SHA1 at release moments. The cases I need the last 16 bytes I should consider an new platform
Again thanks for this inspiration,
This reasoning all seems sound to me. I also agree that if you are down to the last 16 bytes and can't afford space for the UUID, you are probably ready to look for another platform. I think the utility of your solution would be high enough that I'd go find a way to shrink the image by 16 bytes rather than forgo the UUID.
As for the SHA1, I agree with tastewar that if it is a hash of just the executable image, then it won't capture things like updated comments, formatting changes, or even logic changes that compile down to the same opcode stream. If it is always used in conjunction with (and includes in its hash) the UUID, then it would cover trivial source changes. Also, I think you would need to know the range of memory to calculate hash - you wouldn't want to do the full 32K since that may include garbage from prior uploads, and even runtime flash data.
Cheers,