Strange problem with "sprintf"

Hi all,

I am having a strange problem with a little test sketch I wrote in preparation for a larger project.

I’m trying to format strings with “sprintf” so that I don’t need to build up output with multiple print calls.

Here’s my code:

// LCD library
#include <LiquidCrystal.h>

LiquidCrystal lcd(11,10,9,5,4,3,2); // Noritake 16x2 VFD in 4 bit mode

#define chars 16
#define lines 2

char buffer[20]; // sprintf buffer
const char *hex_mask = "ADC5: 0x%03X";
const char *dec_mask = "CAL : %.2f VDC";

unsigned long readAnalog(byte port)
{
        #define reads 1000
        unsigned long value = 0;
        for (int x = 0; x < reads; x++) {
                value += analogRead(port); // multiple reads avg out noise 
        }

        return (value / reads);
}

// floating point version of map()
float fmap(float x, float x1, float x2, float y1, float y2)
{
        return ((x - x1) * (y2 - y1) / (x2 - x1) + y1);
}

void setup()
{
//      pinMode(12, OUTPUT);
//      digitalWrite(12, LOW); // only needed for LCD (Vee)
        lcd.begin(chars, lines);
        analogReference(EXTERNAL); // use external 5.0V reference
}

void loop()
{
        unsigned long vlong = 0;
        float vfloat = 0;

        lcd.setCursor(0, 0); // column 0, row 0

        vlong = readAnalog(A5); // read ADC
        sprintf(buffer, hex_mask, vlong);
        lcd.print(buffer);

        lcd.setCursor(0, 1); // column 0, row 1

        vfloat = fmap(vlong, 0.0, 1023.0, 0.0, 5.0); // map A/D to voltage to 0...5
        sprintf(buffer, dec_mask, vfloat);
        lcd.print(buffer);
}

However, on the VFD (LCD) display, it shows:

ADC5: 0x3FF

← right

CAL : ? VDC

← wrong!

I tested the relevant code with GCC in Linux and it works fine:

#include <stdio.h>

// floating point version of map()
float fmap(float x, float x1, float x2, float y1, float y2)
{
        return ((x - x1) * (y2 - y1) / (x2 - x1) + y1);
}

int main(void)
{
        char buffer[20]; // sprintf buffer
        const char *hex_mask = "ADC5: 0x%03X";
        const char *dec_mask = "CAL : %.2f VDC";
        unsigned long vlong = 1023;
        float vfloat = fmap(vlong, 0.0, 1023.0, 0.0, 5.0);

        sprintf(buffer, hex_mask, vlong);
        fprintf(stdout, buffer);
        fprintf(stdout, "\n"); // needed for screen

        sprintf(buffer, dec_mask, vfloat);
        fprintf(stdout, buffer);
        fprintf(stdout, "\n"); // needed for screen

        return 0;
}

…results in:

root@michael:~/c-progs# ./test
ADC5: 0x3FF
CAL : 5.00 VDC

Anyone see anything wrong? This is driving me nuts! It should work. I suspect it’s a simple problem staring me right in the face…

Any ideas will be appreciated. Thanks!

– Roger

%f isn't included in the Arduino's implementation of sprintf().

Support for vprintf formatting of floating point values is optional in the AVR runtime library and the Arduino does not enable that option, so none of the printf-like functions support floating point formats.

Here is the function that will do the float to string conversion: http://arduino.cc/forum/index.php/topic,72682.msg545115.html#msg545115

Yes, the sprintf implementation used in Arduino seems to have a problem with floats. Not sure if it is code or the wrong module is being linked in or what.

However, the avr-libc function dtostrf() does work and you could use that. It will convert a float to a string, but it cannot include text like with sprintf. So you will have to print the segments separately or use an extra buffer and do the sprintf to insert the string in the text.

The function has to following parameters :
dtostrf(floatVar, minStringWidthIncDecimalPoint, numVarsAfterDecimal, charBuf);

Hope this helps,
Guido

Thank you all for the reply. The function does work (although I don't know what good it is - given that I still have to assemble the pieces into one output string).

I can't imagine why %f isn't implemented. Oh well.........

Thanks again.

-- Roger

krupski: I can't imagine why %f isn't implemented.

The code to handle floats in vfprintf() is large, and the way printf works, it is not until runtime that it can tell if float support is needed, so there is no optimization possible to eliminate it conditionally. The good folks at AVR-GCC and AVR-LIBC cater for this via link-time options to select whether to use the float-aware version or not.

Here in Arduino-land, no one has hooked these options in the underlying technology to anything you can easily control.

dtostre() and dtostrf() are there and work, although they are very fussy to use, by comparison with sprintf(). The printable class has its own brute-force implementation that can't handle %e notation, but will print floats with decimal places.

As suggested, you can use dtostrf(). In terms of your code, you would do it in two stages, first convert the float to a string, then merge the string with everything else:

        const char *dec_mask = "CAL : %s VDC";
        ...
        char floatString[10];
        dtostrf(vfloat,4,2,floatString);
        sprintf(buffer, dec_mask, floatString);

Not directly related, but possibly useful: This will let you use printf rather than sprintf and not need to use a buffer. http://arduino.cc/forum/index.php/topic,120440.0.html

What you can do is create a new AVR libc that has floating point support enabled by default. To do this you have to mess with the libc archive files.

I'm not a fan of the way the avr libc did their vfprintf() floating point support separation.

A bit of background: What they have done is put a vfprintf() in libc that is crippled (no floating point support) to save code space. Then if floating point or an even smaller minimal vfprintf() is wanted, the linker is told to ignore the vfprintf() symbol and then the appropriate library is linked in by specifying it on the command line to the linker.

Because the Arduino team has never supported linker options in the IDE you can not simply change something like a board type to alter the linker option to get the appropriate magic to enable xxprintf() floating point support (You can do this with mpide)

To work around this you can create a modified libc library that has the floating point vfprintf() code in it rather than the non floating point code. To do this, you have to extract and remove the non floating vfprintf module (vfprintf_std.o) from libc.a and then add/move the floating point module (vfprintf_flt.o) from libprintf_flt.a into libc.a

While it is a bit tedious, you only have to do this once. I'd recommend saving the original libc.a and then modify a copy of it. That way you can go back to the original should you want to.

The trick is finding the correct libc.a file as there is is libc.a for each AVR architecture. If you are using linux and Arduino 1.0.1 then the directory you want to go to (for the mega328) is {installdir}/hardware/tools/avrlib/avr/lib/avr5 Note: Prior to Arduino 1.0.1, the Linux IDE did not ship with a compiler and librarys. So if not using Arduino 1.0.1, then you have to locate the avr5 library down in the local system. (This can be different places depending on which release of the tools and which repo you are using)

Once you get down into the avr5 directory: Make a copy of the original libc.a cp libc.a libc.a.orig

You extract & remove the std vfprintf() with: ar -dv libc.a vfprintf_std.o

You extract the floating point vfprintf() with: ar -xv libprintf_flt.a vfprintf_flt.o

You put the floating point vfprintf() into libc.a with: ar -rv libc.a vfprintf_flt.o

After that, you you now have floating point xxprintf() support for the m168/m328, 32u4, and m644 processors by default.

For the Arduino Megas, you will have to modify the libc.a in avr51 and avr6

What I saw on a simple test program that used "%.3f" with the non-floating point code vs the floating point code was about 1600 bytes in code size difference.

--- bill

[quote author=Tom Carpenter link=topic=124809.msg938460#msg938460 date=1348868172] As suggested, you can use dtostrf(). In terms of your code, you would do it in two stages, first convert the float to a string, then merge the string with everything else:

        const char *dec_mask = "CAL : %s VDC";
        ...
        char floatString[10];
        dtostrf(vfloat,4,2,floatString);
        sprintf(buffer, dec_mask, floatString);

[/quote]

HEY! That's an awesome idea! Thanks.

I spent yesterday looking through all the java code of the source (the "Compiler" class) and found where it generates the command line to run the AVR-G++ compiler.

I discovered that there are two library versions... a "printf_min" and a "printf_flt" (and sscan versions) as well as a "libc" and "libm" versions, which of course are small and large (non-floating point vs. floating point) versions of the same libraries.

It is possible to specify that the floating point libraries be used and the "-lm" flag used to link the math libraries, but the compiler command line params are hard coded into the java IDE code.

One thing I've been considering is to add an option to the "preferences.txt" file. For example, if you check on the "Show verbose during compilation" option, the Compiler class has a "verbose ? '-Wall' : '-W'" line in it to use "-Wall" or "-W" on the complier command line depending on if "show verbose" is true or not.

The option I've been considering would be something like "build.use_floating_point=true/false" and then specify the correct command line args depending on the state of the config option.

I can certainly see saving precious memory space, but to LOCK OUT the ability to trade memory for performance is unacceptable.

Sometimes I wonder if actual engineers designed this hardware and software? For example, the mounting holes on the Arduino boards are too close to through-hole components, and there is not enough clearance between board traces and the holes (requiring the use of plastic washers or standoffs in projects).

The software (IDE) is missing a lot of common features such as line cut and paste modes, actual tabs VS spaces, lots of syntax highlight colors hard coded, etc, etc....

It's "open source"... if you have a crowbar to pry it "open" LOL!

Oh well, thanks again for your idea. I'll use it until I figure out how to make the Arduino IDE work properly......

-- Roger

WizenedEE: Not directly related, but possibly useful: This will let you use printf rather than sprintf and not need to use a buffer. http://arduino.cc/forum/index.php/topic,120440.0.html

Thank you. I'll look into it. From first glance, it looks promising. Have to get into it more to be sure.

Thanks again!

-- Roger

bperrybap: What you can do is create a new AVR libc that has floating point support enabled by default. To do this you have to mess with the libc archive files.

Yes I know exactly what you mean. I may just do this.

The only problem is that if a new version of the IDE comes out, I may have to re-hack the code.

The primary thing I use the Arduino boards for is student projects. I am an engineer at the local University here and I support students (in Mechanical Engineering) who need to make senior design projects (usually "assistive devices" to help people who cannot walk or are blind, etc...).

I used to use Motorola 68HC11 "EVBU" boards, but they are larger, cost more and have fewer native I/O ports than the Atmel parts do. So, I switched over to the Arduino.

We build things like motorized lifts or wheelchairs, sonar based distance measuring for blind people, custom remote controls, radio linked devices, etc... all of which need microcontrollers to run the hardware.

If I depend on a "custom" version of the IDE and compiler code, then it will be impossible for others to replicate the projects unless they have the hacked code (we publish the projects with the National Science Foundation).

It would be embarrassing to have someone from another school call me and say "hey your damn thing doesn't work!".

That's a big reason why I need to have code work in "official" IDE releases.

Hopefully the code designers are listening! :)

-- Roger

bperrybap: What you can do is create a new AVR libc that has floating point support enabled by default.

You are the MAN!!! It worked like a champ!

By the way, the original sketch I posted above uses 6952 bytes with FP enabled and 5046 bytes without (1906 bytes more). No big deal when I have 32K on an UNO and 256K on a MEGA.

Lastly, I made your instructions into a little script to make the patch easier. It's a BASH script, but Windoze people should be able to make a BATCH file out of it easily......

EDIT: DON'T USE THIS SCRIPT - USE THE NEW ONE IN [u]POST #16[/u]

#!/bin/bash ################################################################################ # fixfp - script to install floating point support into Arduino printf library # # For more information, see this post: # http://arduino.cc/forum/index.php/topic,124809.msg938573.html#msg938573 ################################################################################

Just change to the directory where the library is (in Linux it's "arduino-1.0.1/hardware/tools/avr/lib/avr/lib/avr5") and run the script. It will patch the library.

Thanks again!

-- Roger

krupski:
It worked like a champ!

I figured it should.
I had just tried it on my own system to make sure to get all the names and paths correct.

Lastly, I made your instructions into a little script to make the patch easier. It’s a BASH script, but Windoze people should be able to make a BATCH file out of it easily…

The issue for the Windoze folks is finding the gnu bin tools
(ar in particular as it is not a native tool for windows)
They actually have all of them, including a bash shell, make, cp, echo, etc…
(they are part of the winAVR package that is shipped with the IDE).
They just don’t know where to find them much less how to use them.
The easiest thing to do (and I have done this in the past) is to write
a batch file that locates the Arduino install area then finds the tools, then
patches the PATH environment to include it, then fires up the shell executable and runs
the very same shell script.
The batch file sets up the environment so you can jump to “*nix” and leave
the DOS/Win world behind to get some real work done.

BTW, there are a few other “patches” that can be done to the AVR core library code
to make it more efficient and smaller to get back most of that code space.
For example the head/tail indexes in the ring_buffer structure in
HardwareSerial.cpp
Given the current code, those need to be unsigned chars vs unsigned ints.
The current code would break if the buffers were ever larger than 256 since the current code
does not properly deal with the atomic issues of multiple byte access (for the int).
By changing it to unsigned chars you can pick up a few hundred bytes of code space
not to mention have faster running code.

— bill

bperrybap: BTW, there are a few other "patches" that can be done to the AVR core library code to make it more efficient and smaller to get back most of that code space.

I would be VERY interested in seeing any and all patches or changes you have. Thank you!

-- Roger

bperrybap: RE: floating point patch

Bill,

I found that the sscanf function also needs to be patched!

Here is an updated script to do the entire patch:

#!/bin/bash
################################################################################
# fixfp - script to install floating point support into Arduino printf library
#
# open software - use, modify, distribute freely
#
# For more information, see this post:
# http://arduino.cc/forum/index.php/topic,124809.msg938573.html#msg938573
################################################################################

STATUS=0

## Exit if libc.a isn't here
test -e libc.a
if [ ${?} -ne 0 ]; then {
    echo "File 'libc.a' not found - exiting"
    exit 0
} fi

test -e libc.a.orig
let STATUS+=${?}

test -e vfprintf_flt.o
let STATUS+=${?}

test -e vfscanf_flt.o
let STATUS+=${?}

## Don't re-apply the patch (would ruin the backup libc.a)
if [ $STATUS -eq 0 ]; then {
    echo "Floating point patch already performed - exiting"
    exit 0
} else {
    cp libc.a libc.a.orig
    ar -dv libc.a vfprintf_std.o
    ar -dv libc.a vfscanf_std.o
    ar -xv libprintf_flt.a vfprintf_flt.o
    ar -xv libscanf_flt.a vfscanf_flt.o
    ar -rv libc.a vfprintf_flt.o
    ar -rv libc.a vfscanf_flt.o
    echo "Floating point patch installed."
} fi

(edit): If you are using a different OS and/or do not wish to patch your files, you may instead download [u]THIS[/u] package (a .ZIP file) which contains the patched versions of "libc.a". Then, locate the "libc.a" files in your Arduino distribution, rename all of them to something like "libc.a.backup" (to preserve the original), then copy the new "libc.a" files from the ZIP archive into your distribution.

Take note of the different versions for different processor directories. They must all go in the proper places! Zipfile contents:

Archive: libc_all.zip Length Name


581202 avr25/libc.a 584430 avr3/libc.a 584474 avr31/libc.a 581702 avr35/libc.a 580574 avr4/libc.a 581058 avr5/libc.a 581106 avr51/libc.a 581218 avr6/libc.a 581490 avrxmega5/libc.a 581650 avrxmega7/libc.a 583910 libc.a


6402814 11 files

You will now have full floating point support for the "sprintf" and "sscanf" functions.

NOTE that this updated library will cause your compiled sketches to be approximately 1500 bytes larger due to the floating point code. You may go back to the original library at any time simply by deleting the patched "libc.a" files and renaming your backups back to "libc.a".

Thanks to bperrybap for the original idea!

-- Roger

krupsi, I would like to use your patch on Windows, how to proceed?

What are those commands, and where to find those files?

cp libc.a libc.a.orig
ar -dv libc.a vfprintf_std.o
ar -dv libc.a vfscanf_std.o
ar -xv libprintf_flt.a vfprintf_flt.o
ar -xv libscanf_flt.a vfscanf_flt.o
ar -rv libc.a vfprintf_flt.o
ar -rv libc.a vfscanf_flt.o

Thanks

guix:
krupsi, I would like to use your patch on Windows, how to proceed?

What are those commands, and where to find those files?

cp libc.a libc.a.orig

ar -dv libc.a vfprintf_std.o
ar -dv libc.a vfscanf_std.o
ar -xv libprintf_flt.a vfprintf_flt.o
ar -xv libscanf_flt.a vfscanf_flt.o
ar -rv libc.a vfprintf_flt.o
ar -rv libc.a vfscanf_flt.o




Thanks

Sorry… I know virtually nothing about Windows. But I assume the AVR libraries are the same for any platform… I attached a ZIP file of all the floating point versions of “libc.a”. You can try them if you like… rename your originals to something like “libc_original.a” then replace them with the new ones in the ZIP file and try it. If it doesn’t work, you can just go back to the originals.

Hope this helps…

– Roger

libc_all.zip (954 KB)

Yes it does :D

Thanks a lot, it's working! Karma++;