Pages: [1] 2 3   Go Down
Author Topic: Option to bypass "Processing -> C++" pre-processor  (Read 4288 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I would like to see an option to turn off/byapps the "Processing -> C++" preprocessor, and instead directly write the main() function,
include my own prototype declarations, header files, calls to any required intialization functions, etc.

You might think "well, just ditch the Arduino IDE and go directly to avr-gcc", but the cost in doing that is that you lose access to the great wealth of Arduino libraries written by third parties, many/most of which rely on the "wiring" Arduino core libs to some degree.

So I'd ideally like to stay in the Arduino library "ecosystem", just opting out of writing "sketches" in "Processing", and instead writing C++ programs directly all the way through. I already write a considerable amount of C++ for my Arduino programs when I decide it's time to package certain things up into an Arduino "library" anyway -- I'd just like the option of doing this all the way through.

I already use my own editor by selecting the "use external editor" feature -- I really appreciate being able to do that.

Why do I wish to be able to disable the "Processing" preprocessor? Mostly because, for me, I find the benefits are far outweighed by the drawbacks. I've been bitten too many times by bugs that have me wasting way too much time chasing nonsensical compiler errors. The inability of the "Processing -> C++" preprocessor to even manage standard C/C++ conditional preprocessor directives is well documented. This makes configuration management a pain, and while the Arduino development team are well aware of the issue, it has been made clear it's not on the list for fixes (sounds like the preprocessor regular expression code has gotten to the stage that no-one fully understands exactly what it's doing anymore) -- so it looks like it's not going to get fixed anytime soon.

But I've come across weirder and harder to understand (and therefore workaround) bugs -- most recently I had some code that would throw up spurious errors (some name collision of some sort going on under the hood, I suspect), but _would_ compile without error when the offending code was placed in a seperate .h file and then inlined in the sketch in the same place via a #include directive.

That just about did it for me.

So I'm just at the stage where I really think I I'd like to leave "Processing" behind me (except perhaps for really quick and dirty apps where the typing time is actually a significant part of the development time.)  

Would anyone else like to see turning off the "Processing -> C++" preprocessing as an option within the Arduino toolchain? I appreciate the simplifications that "Processing" offers over real C++ is a boon for programming novices, but not everyone who programs Arduinos is a programming novice, and one size doesn't fit all. I'd very much like the option to be able to trade off the costs and benefits of using "Processing" on a case by case basis -- seeing an extra checkbox under "Use external editor" that says "Use C++ instead of Processing" would be a very welcome feature indeed.

Alternatively, anyone who has been down this path have any advice on how to proceed sidestepping "Processing" without throwing the Arduino baby out with the bathwater (so to speak)? Thoughts and discussion appreciated.
« Last Edit: July 01, 2012, 02:48:55 am by pico » Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

Global Moderator
Melbourne, Australia
Offline Offline
Brattain Member
*****
Karma: 510
Posts: 19306
Lua rocks!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

You can write your own "main", eg. this compiles:

Code:
int main ()
 {
 return 0;
 }
 

You could make the "processing" sketch (the .ino file) simply have the includes for any libraries you want, and then use .cpp files for the rest of the project. That way the preprocessor's affect is minimal.
Logged

http://www.gammon.com.au/electronics

Please post technical questions on the forum - not to me by personal message. Thanks a lot.

Offline Offline
Edison Member
*
Karma: 19
Posts: 1041
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I recently switched to using a makefile. I use this one: https://github.com/sudar/Arduino-Makefile

This means:

  • No GUI! yay
  • I get to use vim!
  • It automatically links in the arduino libraries and it's easy to tell it to link in other ones like Servo or AFMotor
  • It only addes the #include <Arduino.h> to your sketch
  • It only recompiles needed files
  • You can build, upload, and monitor the serial port all with one command (make upload monitor)[/ii]
The arduino core still provides the main function, but you can still override it with your own.

It would be nice to be able to conditionally include HardwareSerial so you don't have to use the extra bytes for the preinstated HardwareSerial object. Can you force the linker to ignore it if you, for example, #define HardwareSerial_h before you include Arduino.h? it might notice nothing is being used from HardwareSerial.cpp
Logged

Fort Lauderdale, FL
Offline Offline
Faraday Member
**
Karma: 71
Posts: 6144
Baldengineer
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
the cost in doing that is that you lose access to the great wealth of Arduino libraries written by third parties, many/most of which rely on the "wiring" Arduino core libs to some degree.
So, keep using the core library.  Everything needed to compile without the IDE is included with the IDE.
Logged

Capacitor Expert By Day, Enginerd by night.  ||  Personal Blog: www.baldengineer.com  || Electronics Tutorials for Beginners:  www.addohms.c

Ayer, Massachusetts, USA
Offline Offline
Edison Member
*
Karma: 54
Posts: 1857
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I recently switched to using a makefile. I use this one: https://github.com/sudar/Arduino-Makefile

It would be nice to be able to conditionally include HardwareSerial so you don't have to use the extra bytes for the preinstated HardwareSerial object. Can you force the linker to ignore it if you, for example, #define HardwareSerial_h before you include Arduino.h? it might notice nothing is being used from HardwareSerial.cpp
Yeah, I've been thinking about switching from the IDE to using normal Makefiles and emacs.  Thanks for the link.

I haven't looked at the Makefile yet, but the standard way of not bringing in things like HardwareSerial, is to put them in a library such as libhwserial.a with the avr-ar and avr-ranlib commands.  Then use -L<dir> -lhwserial on the link line where <dir> is where you have stored the .a files.

More advanced use would build everything with the -fdata-sections -ffunction-sections, and on the link line add -Wl,--gc-sections.  What this does is put every function and global variable into separate linker sections, and the linker would then eliminate sections that have no references.  So, if you included the Hardware Serial object but no one ever called it, it would be eliminated from the final executable.
Logged

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I recently switched to using a makefile. I use this one: https://github.com/sudar/Arduino-Makefile

I just came back to check on the responses here. In the meantime, I've been hacking away getting Martin Oldfield's Arduino makefile v0.8 running under Win32 using the "make" that comes with MSys 1.0, and his ard-parse-boards perl script using Strawberry perl. It's working now, at least with the basic example sketches I've tested it with.  So Yay! indeed. Emacs + makefiles means no more Processing. :-) Managing my own header files and prototypes is a _very_ small price to pay to have the C/C++ preprocessor working properly again. All simpler than I thought it would be, actually.

It seems that everyone has converged on the makefile solution as the way to go. Kudos to Martin Oldfield and others for their work on this!

Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

More advanced use would build everything with the -fdata-sections -ffunction-sections, and on the link line add -Wl,--gc-sections.  What this does is put every function and global variable into separate linker sections, and the linker would then eliminate sections that have no references.  So, if you included the Hardware Serial object but no one ever called it, it would be eliminated from the final executable.

I had a look at Martin's makefile, and these are the options he uses by default as specified in CPPFLAGS and LDFLAGS. So that's nice.

The final thing I'd like to do at some stage is to get the Arduino "libraries" set-up as true link libraries. Setting up a scheme where each target platform (Uno, Duemil, the two Megas, etc.) would have it's own .a file for each library. Each .a file would get built by make on demand if it id didn't already exist the first time it was needed (or rebuilt if any of the dependencies had been changed, of course).  So you could have

$(ARDUINO_DIR)/libraries/DS1307/uno/DS1307.a
 
and

$(ARDUINO_DIR)/libraries/DS1307/mega2560/DS1307.a

for example.

Seems sensible and obvious. (Unless anyone can think of a reason this won't work.)

Martin Oldfield's scheme in v0.8 builds the .o files in $(SKETCH_DIR)/build-cli/libs/DS1307/DS1307.o for example. Which is consistent with the Arduino IDE approach of compiling Arduino "library" files as if they are just additional source files belonging  to the current project, so I suppose that's why he's gone that way.

Nothing I can't live with for the time being, though.

It's fast. It's simple. It's transparent. And eliminates an entire class of bugs (usually just called "Processing", LOL). Really, what's not to like?

I'm a _much_ happier Arduino camper than I was about 24 hours ago!  :-)
« Last Edit: July 01, 2012, 11:22:58 pm by pico » Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

SF Bay Area (USA)
Offline Offline
Tesla Member
***
Karma: 137
Posts: 6805
Strongly opinionated, but not official!
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
get the Arduino "libraries" set-up as true link libraries.
Is there a reason to do this in addition to the "-ffunction-sections"  "--gc-sections" options?

Do set the Arduino IDE to "verbose" mode and watch what it does before you go off trying to create a makefile from scratch...
Logged

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
get the Arduino "libraries" set-up as true link libraries.
Is there a reason to do this in addition to the "-ffunction-sections"  "--gc-sections" options?

Mostly just to avoid the unnecessary recompilations, I guess. The "make" way of doing things is to rationally manage a build so things don't get recompiled unless they actually need to be, after all. So a little bit of a speed of build benefit, and probably a bit more satisfaction that things are being done "properly", if you will. There is some happiness to be had in having a sharp set of tools. But as you point out, perhaps no difference in terms of what the .hex ultimately looks like.

Do set the Arduino IDE to "verbose" mode and watch what it does before you go off trying to create a makefile from scratch...

Good advice -- I will certainly do this if I proceed. But in the near-term, I'll go with the system as it is, in order to get a bit of confidence it has indeed covered all the bases as well as it appears.
Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

For completeness, I should post a link to Martin Oldfield's web page:

https://github.com/mjoldfield/Arduino-Makefile

there are also appear to be a few forks from this to check out if interested e.g.

https://github.com/peplin/arduino.mk

as well as the one by Sudar mentioned above (neither of which I've played with yet.)

Latest cool thing I've discovered: If you want to use your own main.cpp rather than the default "core" version supplied in the IDE distro, easy peasy: Just create a source file in your sketch directory named main.cpp, and will be used to build main.o rather than the use the "core" source file. If main.cpp isn't found, the default one will be used instead. Not sure if this is intended behaviour or just a fortunate side-effect of the order in which the make recipes are executed, but cool nonetheless.

The only thing that has to be in your main.cpp if you want to use the wiring libs is #include <WProgram.h> (or <Arduino.h>), and call init() before calling setup() (or however you decide to organise your main() function.)

Of course, also include any prototypes if you are calling anything beside init(), setup() and loop(), e,g.:

Code:
#include <WProgram.h>

void setup2();

int main(void)
{
init();

setup2();
    
for (;;)
loop();
        
return 0;
}

(this was my test alternative main.cpp to see if it all works as expected.)

Anyway, this is all too great. Using my own editor and the makefiles, there isn't any need to even fire up the IDE at all (unless I feel the need for a quick and dirty in "Processing" I suppose. But since I usually have emacs running anyway, I can't see this happening very often, somehow...)

Woot!
« Last Edit: July 02, 2012, 10:14:04 pm by pico » Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

SF Bay Area (USA)
Offline Offline
Tesla Member
***
Karma: 137
Posts: 6805
Strongly opinionated, but not official!
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
unless I feel the need for a quick and dirty in "Processing" I suppose
Just a nit: it's NOT "Processing."  Processing is a desktop programming environment based on Java in about the same way that the Arduino IDE is based on C++ (http://processing.org )
I don't think that there's a name for the IDE pre-processing that occurs on your .ino files and whatnot.  "Arduino IDE preprocessing" is probably the most accurate phrase...
Logged

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
unless I feel the need for a quick and dirty in "Processing" I suppose
Just a nit: it's NOT "Processing."  Processing is a desktop programming environment based on Java in about the same way that the Arduino IDE is based on C++ (http://processing.org )
I don't think that there's a name for the IDE pre-processing that occurs on your .ino files and whatnot.  "Arduino IDE preprocessing" is probably the most accurate phrase...


Thanks for the clarification. I thought Arduino had simply adopted "Processing" as their beginners' language for the IDE. Is this just me or this a common misperception? I don't honestly even remember hearing about either "Processing" or "Wiring" before becoming acquainted with Arduinos.

Unfortunately, "Arduino IDE preprocessing" is a bit of a mouthful. So I think I shall adopt the Victorian approach and simply declare "and from this day forth we shall never speak of this again".

 
Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

SF Bay Area (USA)
Offline Offline
Tesla Member
***
Karma: 137
Posts: 6805
Strongly opinionated, but not official!
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

It's a common misconception that "Arduino" is some kind of specialized language, rather than pretty straight C++ with a bit of pre-processing applied.  About all that really happens is automatic prototype generation and concatenation of the .ino files (rather than compiling each separately.)  (Unfortunately, the IDE is a little clumsy about exactly where those prototypes should be added, leading to annoying incompatibilities with some common C preprocessor-directive styles...)
Logged

Offline Offline
God Member
*****
Karma: 32
Posts: 830
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

It's a common misconception that "Arduino" is some kind of specialized language, rather than pretty straight C++ with a bit of pre-processing applied.  About all that really happens is automatic prototype generation and concatenation of the .ino files (rather than compiling each separately.)  (Unfortunately, the IDE is a little clumsy about exactly where those prototypes should be added, leading to annoying incompatibilities with some common C preprocessor-directive styles...)

It also attempts to manage header files for you, which is where I suspect most of the bugs come from.

I was also under the impression it was pretty heavily subsetted version of C++ down to not much more than straight C (which of course id a proper subset of C++). And of course, a "sketch" being limited to one source file. Indeed, about the only hint that C++ is actually the underlying compiler rather than C is that class method reference constructions such as Serial.begin() are allowed. I don't think you can actually define classes within the that-which-must-not-be-named though (although I've not tried, I just went to a creating "library" *.cpp file for that whenever needed.)

FWIW, I've never really been convinced that automatically generating prototypes or managing header file inclusions are that much of a big deal that would constitute much of a benefit to novice programmers. Maybe the strict subsetting is helpful, I don't know. Is it useful or counterproductive to force beginners to put everything into one pde or ino file? Again, I don't know.

In any case, I'm just glad to have it behind me, whatever the hell that was.
Logged

WiFi shields/Yun too expensive? Embeddedcoolness.com is now selling the RFXduino nRF24L01+ <-> TCP/IP Linux gateway: Simpler, more affordable, and even more powerful wireless Internet connectivity for *all* your Arduino projects! (nRF24L01+ shield and dev board kits available too.)

Global Moderator
Melbourne, Australia
Offline Offline
Brattain Member
*****
Karma: 510
Posts: 19306
Lua rocks!
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I mention in the sticky here that the underlying language is C++. I think it's an important point to make (which is why it is point #1).

With the exception of exceptions (lol) most other things can be done, and done easily. For example, you can use the STL if you download Andy Brown's port of it (The Standard Template Library (STL) for AVR with C++ streams ).

If you can use that (which relies heavily on templates) you can do pretty-much anything.
Logged

http://www.gammon.com.au/electronics

Please post technical questions on the forum - not to me by personal message. Thanks a lot.

Pages: [1] 2 3   Go Up
Jump to: