Option to bypass "Processing -> C++" pre-processor

WizenedEE:
I recently switched to using a makefile. I use this one: GitHub - sudar/Arduino-Makefile: Makefile for Arduino sketches. It defines the workflows for compiling code, flashing it to Arduino and even communicating through Serial.

I just came back to check on the responses here. In the meantime, I've been hacking away getting Martin Oldfield's Arduino makefile v0.8 running under Win32 using the "make" that comes with MSys 1.0, and his ard-parse-boards perl script using Strawberry perl. It's working now, at least with the basic example sketches I've tested it with. So Yay! indeed. Emacs + makefiles means no more Processing. :slight_smile: Managing my own header files and prototypes is a very small price to pay to have the C/C++ preprocessor working properly again. All simpler than I thought it would be, actually.

It seems that everyone has converged on the makefile solution as the way to go. Kudos to Martin Oldfield and others for their work on this!

MichaelMeissner:
More advanced use would build everything with the -fdata-sections -ffunction-sections, and on the link line add -Wl,--gc-sections. What this does is put every function and global variable into separate linker sections, and the linker would then eliminate sections that have no references. So, if you included the Hardware Serial object but no one ever called it, it would be eliminated from the final executable.

I had a look at Martin's makefile, and these are the options he uses by default as specified in CPPFLAGS and LDFLAGS. So that's nice.

The final thing I'd like to do at some stage is to get the Arduino "libraries" set-up as true link libraries. Setting up a scheme where each target platform (Uno, Duemil, the two Megas, etc.) would have it's own .a file for each library. Each .a file would get built by make on demand if it id didn't already exist the first time it was needed (or rebuilt if any of the dependencies had been changed, of course). So you could have

$(ARDUINO_DIR)/libraries/DS1307/uno/DS1307.a

and

$(ARDUINO_DIR)/libraries/DS1307/mega2560/DS1307.a

for example.

Seems sensible and obvious. (Unless anyone can think of a reason this won't work.)

Martin Oldfield's scheme in v0.8 builds the .o files in $(SKETCH_DIR)/build-cli/libs/DS1307/DS1307.o for example. Which is consistent with the Arduino IDE approach of compiling Arduino "library" files as if they are just additional source files belonging to the current project, so I suppose that's why he's gone that way.

Nothing I can't live with for the time being, though.

It's fast. It's simple. It's transparent. And eliminates an entire class of bugs (usually just called "Processing", LOL). Really, what's not to like?

I'm a much happier Arduino camper than I was about 24 hours ago! :slight_smile:

get the Arduino "libraries" set-up as true link libraries.

Is there a reason to do this in addition to the "-ffunction-sections" "--gc-sections" options?

Do set the Arduino IDE to "verbose" mode and watch what it does before you go off trying to create a makefile from scratch...

westfw:

get the Arduino "libraries" set-up as true link libraries.

Is there a reason to do this in addition to the "-ffunction-sections" "--gc-sections" options?

Mostly just to avoid the unnecessary recompilations, I guess. The "make" way of doing things is to rationally manage a build so things don't get recompiled unless they actually need to be, after all. So a little bit of a speed of build benefit, and probably a bit more satisfaction that things are being done "properly", if you will. There is some happiness to be had in having a sharp set of tools. But as you point out, perhaps no difference in terms of what the .hex ultimately looks like.

westfw:
Do set the Arduino IDE to "verbose" mode and watch what it does before you go off trying to create a makefile from scratch...

Good advice -- I will certainly do this if I proceed. But in the near-term, I'll go with the system as it is, in order to get a bit of confidence it has indeed covered all the bases as well as it appears.

For completeness, I should post a link to Martin Oldfield's web page:

there are also appear to be a few forks from this to check out if interested e.g.

as well as the one by Sudar mentioned above (neither of which I've played with yet.)

Latest cool thing I've discovered: If you want to use your own main.cpp rather than the default "core" version supplied in the IDE distro, easy peasy: Just create a source file in your sketch directory named main.cpp, and will be used to build main.o rather than the use the "core" source file. If main.cpp isn't found, the default one will be used instead. Not sure if this is intended behaviour or just a fortunate side-effect of the order in which the make recipes are executed, but cool nonetheless.

The only thing that has to be in your main.cpp if you want to use the wiring libs is #include <WProgram.h> (or <Arduino.h>), and call init() before calling setup() (or however you decide to organise your main() function.)

Of course, also include any prototypes if you are calling anything beside init(), setup() and loop(), e,g.:

#include <WProgram.h>

void setup2();

int main(void)
{
	init();

	setup2();
    
	for (;;)
		loop();
        
	return 0;
}

(this was my test alternative main.cpp to see if it all works as expected.)

Anyway, this is all too great. Using my own editor and the makefiles, there isn't any need to even fire up the IDE at all (unless I feel the need for a quick and dirty in "Processing" I suppose. But since I usually have emacs running anyway, I can't see this happening very often, somehow...)

Woot!

unless I feel the need for a quick and dirty in "Processing" I suppose

Just a nit: it's NOT "Processing." Processing is a desktop programming environment based on Java in about the same way that the Arduino IDE is based on C++ (http://processing.org )
I don't think that there's a name for the IDE pre-processing that occurs on your .ino files and whatnot. "Arduino IDE preprocessing" is probably the most accurate phrase...

westfw:

unless I feel the need for a quick and dirty in "Processing" I suppose

Just a nit: it's NOT "Processing." Processing is a desktop programming environment based on Java in about the same way that the Arduino IDE is based on C++ (http://processing.org )
I don't think that there's a name for the IDE pre-processing that occurs on your .ino files and whatnot. "Arduino IDE preprocessing" is probably the most accurate phrase...

Thanks for the clarification. I thought Arduino had simply adopted "Processing" as their beginners' language for the IDE. Is this just me or this a common misperception? I don't honestly even remember hearing about either "Processing" or "Wiring" before becoming acquainted with Arduinos.

Unfortunately, "Arduino IDE preprocessing" is a bit of a mouthful. So I think I shall adopt the Victorian approach and simply declare "and from this day forth we shall never speak of this again".

It's a common misconception that "Arduino" is some kind of specialized language, rather than pretty straight C++ with a bit of pre-processing applied. About all that really happens is automatic prototype generation and concatenation of the .ino files (rather than compiling each separately.) (Unfortunately, the IDE is a little clumsy about exactly where those prototypes should be added, leading to annoying incompatibilities with some common C preprocessor-directive styles...)

westfw:
It's a common misconception that "Arduino" is some kind of specialized language, rather than pretty straight C++ with a bit of pre-processing applied. About all that really happens is automatic prototype generation and concatenation of the .ino files (rather than compiling each separately.) (Unfortunately, the IDE is a little clumsy about exactly where those prototypes should be added, leading to annoying incompatibilities with some common C preprocessor-directive styles...)

It also attempts to manage header files for you, which is where I suspect most of the bugs come from.

I was also under the impression it was pretty heavily subsetted version of C++ down to not much more than straight C (which of course id a proper subset of C++). And of course, a "sketch" being limited to one source file. Indeed, about the only hint that C++ is actually the underlying compiler rather than C is that class method reference constructions such as Serial.begin() are allowed. I don't think you can actually define classes within the that-which-must-not-be-named though (although I've not tried, I just went to a creating "library" *.cpp file for that whenever needed.)

FWIW, I've never really been convinced that automatically generating prototypes or managing header file inclusions are that much of a big deal that would constitute much of a benefit to novice programmers. Maybe the strict subsetting is helpful, I don't know. Is it useful or counterproductive to force beginners to put everything into one pde or ino file? Again, I don't know.

In any case, I'm just glad to have it behind me, whatever the hell that was.

I mention in the sticky here that the underlying language is C++. I think it's an important point to make (which is why it is point #1).

With the exception of exceptions (lol) most other things can be done, and done easily. For example, you can use the STL if you download Andy Brown's port of it (The Standard Template Library (STL) for AVR with C++ streams ).

If you can use that (which relies heavily on templates) you can do pretty-much anything.

So you can define your own classes in a pde/ino file? As I say, I never tried, but I was under the impression you had to create a seperate .cpp file to do that.

Whatever it is, it really aint C++ if you can't define classes.

Well you can, as this illustrates:

typedef class
  {
  public:
    int bar;
  } tFoo;

tFoo a;
 
void handler ()
  {
  a.bar = 42;  
  }
  
void setup ()
{
  handler ();
}

void loop () {}

However there is a problem if you try to pass your own defined class to a function, because of the automatic function prototype generation, which puts the prototypes before where the type is defined.

I just raised a bug report about that a few minutes ago:

http://code.google.com/p/arduino/issues/detail?id=973

On the bright side, if I may say without sounding pompous, most people who define a class in C++ would put it into its own .h files. That is really where class definitions belong. Then they can be shared between multiple .cpp files. You can do that under the Arduino IDE (making a new tab), it compiles correctly, and is easy to work with.

Whatever it is, it really aint C++ if you can't define classes.

Yep, you can define classes.

OK, I suppose I should have said define and use. LOL.

Sort of like a write-only memory device. :wink:

In any case, I'm glad to be in C++. No questions of "pretty much" or "bug reports".

I would disagree. Or say that if indeed most people are doing it this way, they are doing it wrong. You put the declarations (basically, prototypes for the class methods) into a .h file, but the the source for the methods is in a .cpp file. You definitely don't want this included in every file that simply want to use a class!

pico:
OK, I suppose I should have said define and use. LOL.

My example used the class. It compiles.

You put the declarations (basically, prototypes for the class methods) into a .h file, but the the source for the methods is in a .cpp file. You definitely don't want this included in every file that simply want to use a class!

I don't get your point here. You put the definition in a .h file. You can put the implementation into the .ino file, or make another .cpp tab for it. Probably neater to put it into a separate file. So say you want implement class foo, you make foo.h and foo.cpp. What is the major problem with that?

In any case, I'm glad to be in C++. No questions of "pretty much" or "bug reports".

Right. So C++ has no bug reports?

pico:
But I've come across weirder and harder to understand (and therefore workaround) bugs -- most recently I had some code that would throw up spurious errors (some name collision of some sort going on under the hood, I suspect), but would compile without error when the offending code was placed in a seperate .h file and then inlined in the sketch in the same place via a #include directive.

That just about did it for me.

Perhaps you should calm down and post the code in question, and try to understand the techniques required to work around any issues you have. The IDE has some idiosyncrasies, designed to make things easier for beginners. Now you can throw out the whole lot, and move to a pure avr-gcc environment, or keep the friendliness of the IDE and develop strategies for avoiding the issues that friendliness causes.

Let me put it to you like this. If you move away from the IDE, perhaps use a custom Make file, or use the Atmel Studio itself, you will no doubt eliminate some of the things that are bugging you. But, and this is important, you will replace them with other things that will annoy you. There is no perfect development environment.

No, of course not. But as there are degrees of evil, there are degrees of imperfection.

I probably should say, just to establish a bit of context, that I've dealt with many, many different development environments over the years, ranging over all sorts of hardware. The Arduino IDE was in no way the worst I've dealt wth. But very far from the best as well. (In fact, to be honest, it really does rank down there in the suckiness stakes. But I have dealt with worse.)

On a more positive note, there are many things apart from the IDE that I like about the Arduinos very much. I love the bootloader and the plug-and-play USB programming model. (I grew up with the old school development boards that you spent about a week getting set-up to get to "hello world" with. Intel, then Motorola, then MIPs.) I like the Atmel mega chips. I love the wealth of 3rd party libraries that means that just about any device you can think of interfacing with an Arduino probably has at least some basic code already available for it.

Which is why, overall, I am an Arduino fan.

But I am not a programming novice. I've been programming in C before there was even ANSI C, much less C++. Arduinos are for beginners. Fine. But I see no reason they have to be only for beginners.

So no, I don't expect "perfect" in a development environment. But emacs and gcc and make is pretty damned good, IMHO -- at least it's always something I've gotten along with. That's not to say "perfect". But, as they say, even the Sun has its spots. :slight_smile:

I'm glad there are options for those with differing tastes. Perhaps we can agree to agree on this.

pico:
But I am not a programming novice. I've been programming in C before there was even ANSI C, much less C++. Arduinos are for beginners. Fine. But I see no reason they have to be only for beginners.

Speaking as somebody who has used C since 1977 or 1978 and was on the original ANSI X3J11 C standards committee, I tend to agree with you in terms of the IDE trying to simplify and protect users. But as you point out not everybody who programs Arduinos needs the training wheels. In terms of development environments, I tend to be old school, and still haven't warmed to any graphical IDE, and only use it as a compilation unit and bootloader.

Found the definitive workaround, as discussed earlier in the thread.

You find the IDE "friendly". I find it a buggy PITA. Let's agree to disagree. And be happy for each other that you can use the IDE, and I can choose not to.

MichaelMeissner:
In terms of development environments, I tend to be old school, and still haven't warmed to any graphical IDE, and only use it as a compilation unit and bootloader.

I suspect our tastes are similar, and I'm now very happy to report that I now don't even require the IDE for compilation or bootloading. Recommended.

BTW, good job on ANSI C.