next topic »

[on:]

Hi all,

In relation to some ideas around regression testing I began looking at adapting the preprocessing/compilation portion of the Arduino IDE so that it could be run from the command line.

Under the assumption it would be easier (and more fun) to be using Python rather than Java but still wanting to leverage the existing Java code I began to explore using Jython.

I've documented my experience so far with the Arduino IDE and Jython. This is post is a summary of my findings so far.

I've managed to run the pre-preprocessor successfully on a sketch from the commandline with a very short Jython script.

With the preprocessor accessible in this manner it should be possible to modify the Makefile to work on a standard PDE file I think. I presume it would also be possible to make this behaviour work as a command line option for the IDE with some work.

Unfortunately, when I moved on to trying to get the example compiled as well, I discovered the IDE is highly coupled—even between UI and non-UI-specific classes. The source also has a lot of code (sometimes commented out, sometimes not) that is only required for its original use in the Processing IDE. Many of the comments have not been changed from the Java-specific comments of the Processing IDE to the C++ ones of the Arduino IDE.

The biggest issue is that you need to have a Sketch instance and to create one you need to supply it an Editor instance—which pulls in GUI related code. The Editor class accesses Base and Preferences classes as well, which then brings in dependencies on preferences files, file paths and operating systems and it all blows up in a mess.

The sketch shouldn't require an Editor instance in order to compile it. I tried supplying a Null but it barfs. I then tried sub-classing the Editor class in order to dummy out the routines causing problems, but there seems to be no way to stop the original/super constructor from being called.

So, it seems without extensive re-working getting the compilation stage to work independently of the IDE won't be a goer. IMO the two should be totally separated but I suspect it would be an uphill battle trying to change the IDE code if there's a desire to stay inline with the Processing IDE.

I think there would be a lot of benefit in having the GUI code de-coupled from the preprocessing/compilation stages in terms of development and testing. I must admit I am skeptical about the integration work required to make this happen being supported by the core Arduino and Processing teams however given other, more obvious, demands. But I could be surprised. :-)

None the less, I'm pleased it at least seems possible to get the pre-processor to run separately and will continue poking at this as required by progress with the testing side of my project.

Feedback?

--Phil.

[Also posted to the Developers list.]

[Reply #1 on:]

Interessting. Would love to try it.

What I don't like is that you use Jython. When I modify the IDE, then this is all Java. I don't like Java, but at least everything is written in a single programming language at the moment.
Now it seems as if I would have to use another programming language for some parts. Maybe this is a benefit but I can't see it at the moment.

[Reply #2 on:]

Now it seems as if I would have to use another programming language for some parts. Maybe this is a benefit but I can't see it at the moment.

Heh, I understand that. :-) The benefit is purely to me, in that I managed to test out an idea in a language I prefer to use but without having to reinvent the wheel.

This was entirely intended as a proof-of-concept exercise. However, AFAIK it is possible to to create a standard standalone .jar file of a Jython script so it should be possible to be able to run the script without separately installing Jython.

You are more than welcome to translate this from a proof-of-concept to something either integrated into the IDE binary, or a separate Java command line tool--the later I don't imagine would be too difficult. (Famous last words.)

--Phil.

[Reply #3 on:]

Hi Phil,
thanks for sharing your work. I really like the idea of creating a set of classes for proccessing/compiling(/uploading?) sketches.
This would offer a wide range of use-cases. First things I can think of is creating the obvious commandline-tools and creating an Sketch-plugin for Ant.

As for decoupling the arduino-GUI from the build-process:
I get the impression that GUI-issues are a rather slow moving target.
The Linux version of the IDE must be the last application on earth that still plagues its users with the original non-Swing FileDialogs. They are so retro (ugly and alien to every modern Linux-Desktop) that they will become mega-hip again in two years from now!

Since the preprocessing-stage seems to be already solved, I had a look at the sources of Compiler.class and the Sketch.class it depends on.

My first impression is that the references made to the Editor.class are mainly readonly. So there is no real interaction with the Editor.class.

The code seems  to query things like library-locations and sketch-file names or the target board.
It should be possible for a non-GUI Compiler.class to get the same information without going through an instance of class Editor.    
Sketches are pretty well organized, since they all live only in a single directory of their own.
For the rest of the things to know (user preferences etc.) for compiling we should be able to find out ourselves as long as we know where the arduino root directory is.

If you (or anybody else ) are interested in creating a set of commandline tools, I would happily join the team.

Eberhard