My top 5 wish list for improvements to the Arduino core library/IDE

These are all based on real needs. It appears to me that none of these is implemented in 1.0 beta.

1. Additional member functions print_P and println_P in the Print class, for printing PROGMEM strings

I often see posts on the forum where users report that their sketches don't work since they added a small piece of apparently benign code. The usual reason is running out of RAM, and the most common reason for running out of RAM is use of a large number of string literals. The fix is to move the strings into PROGMEM, however they can't then be printed directly using e.g. Serial.print(). I suggest new member functions print_p(const PROGMEM char*) and println_p(const PROGMEM char*) in the Print class, similar to the corresponding print and println members but taking PROGMEM strings. This would allow for example:

Serial.println_p(PSTR("Hello world!"));

The _p suffix is in line with other functions that take PROGMEM strings. I have code available for these functions.

[EDIT: in fact the 1.0 release includes a new F() macro for creating strings in PROGMEM, and print/println are overloaded to support it - so print_p and println_p are not required. Thanks to CodingBadly for poinitng this out.]

2. Facility for hooking the tick interrupt

Many sketches need a regular tick interrupt, for example for polling buttons and rotary encoders. I usually use the MsTimer2 library to set up a tick interrupt. However, there is already a regular interrupt used to count millseconds, and it would make sense to hook into this instead, keeping timer 2 free for other uses. See http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1293546475 for a possible implementation.

3. When compiling a sketch, as well as displaying the amount of Flash used, display the amount of RAM used by static data and string literals.

The IDE currently provides no clues to the user that lack of RAM may be an issue. In fact, lack of RAM is much more likely to be a problem than lack of Flash. See http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1224729260/all for a possible implementation.

4. Configurable delay in analogRead() after switching the multiplexer

Another problem that I often see on the forums is users finding that analog pins 'interfere' with each other. The usual suggestion is to read from the pin twice, possibly with a delay between the two reads. However, it would be better to include a configurable delay in analogRead() itself, for two reasons:

(a) including a small amount of delay by default would avoid many users experiencing the problem in the first place;

(b) a delay is more effective than doing multiple analogRead() calls, i.e. a stable reading is achieved in a shorter time.

See Delay in analogRead after switching pin - #13 by dc42 - Libraries - Arduino Forum for more details and some measurements I took. The default delay of 10uS that I suggest adds less than 10% to the time taken by an analogRead() call. As an alternative to the suggestion I made in that post, the delay could be passed as an optional second parameter to analogRead().

5. Fix the bug that causes delayMicros(0) to delay for a very long time

This is the sort of bug that may go unnoticed, until an unsuspecting user calls delayMicros with a calculated delay time and the calculation just happens to return zero on rare occasions. See BUG: delayMicroseconds(0) leads to a huge delay - #8 by robtillaart - Libraries - Arduino Forum.

KE7GKP:
Re: (4) Note that the analogRead() anomaly is dependent on the source impedance of the analog voltage. The "interference" (crosstalk) and "delay" (settling time) issues are diminished or eliminated by a lower impedance. Hard-wiring factors based on the assumption of marginal circuit design doesn't seem like a desirable thing to me. Like many design decisions, it is an interaction of several factors, hardware and software.

Yes, the analogRead() problem goes away with sufficiently low source impedance. However, I don't regard using source resistances of greater than 10k as marginal circuit design when the signal is varying very slowly, for example, from a temperature sensor (sampling an audio signal would be a different matter). And I'm not proposing to hard-wire anything, my proposal is for a configurable delay with a fairly low default value.

Presumably, whenever a user reports this issue and is using a sensor with a source resistance of more than 10k, you think we should always tell him/her to add an op-amp to buffer the input pin, rather than use the workaround of reading the pin more than once?

Thanks. I completely agree with you that it must be easy to reduce the delay to zero. Also, any default delay must be small enough to have minimal impact on existing sketches.

My objectives behind this proposal are:

  1. Reduce the amount of trouble that Arduino newbies have; and

  2. Provide a more efficient (time-wise) way of handling source resistance greater than 10K. For example, my measurements indicate that with 100k source resistance, a delay of 10us is sufficient, as opposed to doing a second analogRead call which takes over 100us.

dc42:

  1. Additional member functions print_P and println_P in the Print class, for printing PROGMEM strings

void setup( void )
{
Serial.begin( 57600 );
}

void loop( void )
{
Serial.println( F( "This is a test." ) ); // <<<
delay( 1000 );
}

The fix is to move the strings into PROGMEM, however they can't then be printed directly using e.g. Serial.print(). I suggest new member functions print_p(const PROGMEM char*) and println_p(const PROGMEM char*) in the Print class, similar to the corresponding print and println members but taking PROGMEM strings.

Does not work. The compiler cannot tell the difference between "const char*" and "const PROGMEM char*".

dc42:
4. Configurable delay in analogRead() after switching the multiplexer

Another problem that I often see on the forums is users finding that analog pins 'interfere' with each other. The usual suggestion is to read from the pin twice, possibly with a delay between the two reads. However, it would be better to include a configurable delay in analogRead() itself, for two reasons:

Ideally, analogRead should provide both the delay you recommend and discarding-the-next-value recommended by Atmel.

For each read, ADC Noise Reduction Mode and averaging / filtering would also be useful additions.

This would make a very nice library. :wink:

You appear to have misunderstood my proposal. The reason that member functions with new names are needed (i.e. print_p and println_p, with the _p suffix) is precisely because the compiler can't tell the difference between "const char*" and "const PROGMEM char*", and therefore it is not possible to achieve the required functionality by overloading print and println.

It is indeed unfortunate that 'print(PSTR("Hello world!"))' can't be made to either work or give a compile-time error, neither for that matter can 'print_p("Hello world!")'. The onus will be on the user to use print_p and println_p only with PROGMEM strings. Nevertheless, having members of the Print class that will print PROGMEM strings directly would be a big improvement on the current situation.

Nevertheless, having members of the Print class that will print PROGMEM strings directly would be a big improvement on the current situation.

And you appear to have misunderstood the point of my posting the code snippet.

Apparently I have. When you said "Does not work", what exactly did you mean? Would you like me to post my proposed, tested implementation of print_p and println_p?

You appear to have misunderstood my proposal

I read through your proposal too quickly. I apologize.

When you said "Does not work", what exactly did you mean?

Overloading print[ln] to support PROGMEM. But that was not your proposal.

And you appear to have misunderstood the point of my posting the code snippet ... Apparently I have.

The new F-macro creates an inline PROGMEM string constant. print[ln] supports printing F-strings.

Excellent! That's one less thing to have to workaround when Arduino 1.0 is released. And the implementation is so simple too.

No it does not.The macro creates a flash string object.

Not true (as of 0022), but print[ln] can be modified to to so by adding new overloaded methods.

Are you saying that the flash class is included with release 1.0 or are you suggesting it should be?

BenF:

[quote author=Coding Badly link=topic=73605.msg553741#msg553741 date=1317148451]
The new F-macro creates an inline PROGMEM string constant.

No it does not.The macro creates a flash string object.[/quote]

No, it creates an inline PROGMEM string constant...

#define PSTR(s) ((const PROGMEM char *)(s))

class __FlashStringHelper;
#define F(string_literal) (reinterpret_cast<__FlashStringHelper *>(PSTR(string_literal)))

__FlashStringHelper has no definition. It is used much like an anonymous structure would be used.

BenF:

[quote author=Coding Badly link=topic=73605.msg553741#msg553741 date=1317148451]
print[ln] supports printing F-strings.

Not true (as of 0022), [/quote]

I believe the topic is Arduino 1.0.

but print[ln] can be modified to to so by adding new overloaded methods.

Already done (with thanks to Mikal Hart).

Are you saying that the flash class is included with release 1.0 or are you suggesting it should be?

I'm saying the F-macro is present in the Arduino 1.0 beta and that the string constant is placed in Flash in the same way that PROGMEM places a string constant in Flash.

dc42:
These are all based on real needs. It appears to me that none of these is implemented in 1.0 beta.

  1. ...
  2. ...
  3. ...
  4. ...
  5. ...

Thumbup for suggestions :wink:

If you change above to PROGMEM I can agree.

The F macro will return a class reference, but fortunately this will not add overhead in terms of larger pointer size (frame + member reference) as the class reference is 16-bit wide in this case. It's a simple cast from/to a PSTR (also 16-bit wide) and so generated code will be the same as for a direct print_P implementation. For me either would work just fine, but I think they should consider a better name for the macro. Also I don't quite see the need to depart from the _P convention already known and apreciated by many users.

Thanks for this list. While I have pretty much no control over the official Arduino IDE and environment, I'm always looking for ideas to incorporate into Teensyduino.

Teensyduino already has #1 and #5 for, for 0022 and earlier, and also 1.0-rc1.

The F() macro and flash string support appearing in 1.0 was contributed by me, with help and input from Mikal Hard and Brian Cook, both on the Arduino developers list and in private followups. I realize the _P suffix is commonly used by avr-libc and other programs. It was discussed, but there was strong consensus it wasn't Arduino's API style. Had I gone with a _P suffix, this contribution never would have been accepted (it took many months as-is to convince the Arduino Team to merge it).

Something they did not accept, but I tried to contribute, was F() flash-string integration with the String class. So you can use F() with XYZ.print(), but not with anything that takes the newer String objects.

Teensyduino has a heavily optimized delayMicroseconds(), which works for 0 and also small values, with exactly cycle-accurate delays for the common const input case. If anyone wants to see it in Arduino, the code is available. I doubt it ever be accepted, because my coding style and extreme use of inline assembly is not consistent with Arduino's coding style. But if you want this fixed, it is indeed fixed in Teensyduino and the code is open source (after installing, look in hardware/teensy/cores/teensy).

#3 is on my serious wish-list. I worked on some elf/drawf parsing code to extract the call tree and stack framing info from the debug data inside the .elf file. It's quite tricky, and of course if recursion is detected you can't know worst-case stack use. I may never get this written, but I really, really want to have this feature, since running out of RAM really sucks.

I'm also really interested in #2, but if I develop it for Teensyduino and Arduino proper never accepts it, I wonder how much use it would ever get? The huge benefit I see is for libraries. I was considering a table of perhaps 5 or 10 callbacks, which could be triggered at something like 100 Hz (so with 10, the ISR would never need to call more than one). That way, if the user needs 1 or 2, and uses a couple libraries than each need 1 tick callback, everything would still work nicely.

I'm open to ideas on this, and there's a very good chance I'll implement it in Teensyduino at some point, but my hope is the Arduino developers can at least buy into the idea and commit to an outline of the API... so it doesn't end up being a contribution they'll never accept. (while many things I've contributed are now part of Arduino, on average most of the stuff I try to contribute never gets used... eg, look at issue #140).

Anyway, my point of this lengthy rambling post is I am indeed working on some of this stuff for Teensyduino, and a couple have been available for over a year. I generally do try to contribute my improvements back to Arduino when they're general to the hardware, but usually it takes a very long time for any contribution to be accepted into Arduino proper, and often they just sit unused indefinitely.

Hi Paul, thanks for your feedback.

That's good to know.

Thanks for your contribution! I'm happy with the F() macro. It has the advantage that selection of the correct overload is automatic. With print_p and println_p there would always be the risk of users calling them with non-PROGMEM strings, or calling print and println with PROGMEM strings. As long as users use F() instead of PSTR() to create PROGMEM strings, that risk is gone.

I won't cry over this, because IMO anything that uses dynamic memory allocation other than during initialization is a bad idea on an embedded platform with limited RAM, because of the memory fragmentation and unpredicatable RAM usage it causes. I appreciate that String makes some operations a lot easier to code, but I consider that the risks of using it are too high.

While it would be very nice to be able to predict the total RAM use including the stack, I'd settle for just knowing the amount of RAM used by static data and string literals, so that I can see how much is left over for the stack.

Yes, I suppose that if it were implemented for Arduino, libraries would start using it, so it needs to be chainable. Rather than a fixed table, I would define a link structure holding a pointer to the callback function and a pointer to the next link, and require anything that wants a callback to provide one of these structures. However, a 100Hz callback frequency is too slow for some purposes, e.g. multiplexing more than two or three 7-segment displays without flicker, or polling rotary encoders (I have found that polling rotary encoders at 1ms intervals works well, but if the interval is increased to 4ms then they don't work if you twist them quickly).

Yes, it's frustrating when good ideas don't get accepted and incorporated. Thanks for all your efforts!

Thanks to dc42 for starting an interesting discussion. Is the F() macro similar to the one in Mikal Hart's "Flash" library? And are there any other parts of this library which have been incorporated into the 1.0 Arduino core?

  1. Additional member functions print_P and println_P in the Print class, for printing PROGMEM strings

fixed, apparently.

  1. Facility for hooking the tick interrupt

Expensive and rather dangerous. The tick interrupt is already too long, and calling an external function would immediately add significant overhead when the ISR code has to save extra state not normally saved by function calls. I like the idea of having a separate periodic function, but tying into the existing timer tick seems like a bad way to do it.

  1. When compiling a sketch, as well as displaying the amount of Flash used, display the amount of RAM used by static data and string literals.

There are several google code issues opened with respect to this.
http://code.google.com/p/arduino/issues/detail?id=40
http://code.google.com/p/arduino/issues/detail?id=395
http://code.google.com/p/arduino/issues/detail?id=444
Apparently part of the issue is that the amount of AVAILABLE ram is unknown to the IDE, so for the beginner audience this would just be a meaningless number printed out.

  1. Configurable delay in analogRead() after switching the multiplexer

Perhaps the existing code should just work? Or is this one of those things that is heavily dependent on the impedance of the analog source?

  1. Fix the bug that causes delayMicros(0) to delay for a very long time

I guess. I'm not sure that coming up with 0 as a result of calculations is that much more common than coming up with a negative number, which would also cause an unexpectedly long delay...

Hi @westfw, thanks for your feedback.

westfw:

  1. Facility for hooking the tick interrupt

Expensive and rather dangerous. The tick interrupt is already too long, and calling an external function would immediately add significant overhead when the ISR code has to save extra state not normally saved by function calls. I like the idea of having a separate periodic function, but tying into the existing timer tick seems like a bad way to do it.

From what perspective is the tick interrupt already too long, and in what way is hooking it dangerous? Surely it depends on whether or not you are using e.g. delayMicros() to do some critical timing?

I can see the validity of your point about overhead to save extra state, assuming that the compiler currently saves only the registers actually used in the ISR. Would it not be possible to use the existing ISR initially, and change the tick interrupt vector when the sketch makes the call to request a callback on each tick? That way, the time taken by the tick interrupt would not increase from the present until someone wants to hook it.

westfw:

  1. When compiling a sketch, as well as displaying the amount of Flash used, display the amount of RAM used by static data and string literals.

Apparently part of the issue is that the amount of AVAILABLE ram is unknown to the IDE, so for the beginner audience this would just be a meaningless number printed out.

I can't see any excuse for not adding this information to the boards.txt file.

westfw:

  1. Configurable delay in analogRead() after switching the multiplexer

Perhaps the existing code should just work? Or is this one of those things that is heavily dependent on the impedance of the analog source?

Yes, it is heavily dependent on the resistance of the analog source. With no delay, the voltage on the previous pin read starts affecting the result of analogRead() when the source resistance is above 10K. With 10us delay (compared to >100us conversion time), my measurements indicate that there is no interference even at 100K source resistance.

westfw:

  1. Fix the bug that causes delayMicros(0) to delay for a very long time

I guess. I'm not sure that coming up with 0 as a result of calculations is that much more common than coming up with a negative number, which would also cause an unexpectedly long delay...

Maybe not, but the behaviour when passing 0 violates the reasonable expectations of the user.

Would it not be possible to use the existing ISR initially, and change the tick interrupt vector

no, the vectors are all in flash and can't be changed at runtime.