Go Down

Topic: How to safely and reasonably convert a float or double to string or char array? (Read 2077 times) previous topic - next topic

BulldogLowell

You are not correct.

There are things that may be counted that may not be represented in decimal form. 

I'm not typing into a 64.127894 bit computer.

There isn't even a need for a decimal when using the set of integers.


Delta_G

There isn't even a need for a decimal when using the set of integers.


There isn't a need sure, but that doesn't mean that the number isn't that exact.  Again, you're talking about how you would represent that number.  Whether or not you should WRITE a decimal point.  But the number itself is the same number even if you do. 

Are you telling me that 64 != 64.00?
If at first you don't succeed, up - home - sudo - enter.

BulldogLowell

In the set of integers, 64.00 does not exist. You are comparing a number against a non-existent/undefined thing.

Just like 1.0/0 does not equal infinity.

Take a freshman level number theory class or just read the text and you'll understand.

Not withstanding all of this, OP is really off the reservation with his/her expectations of a floating point number on a 16bit platform (even a 64 bit platform!).

Delta_G

In the science world we actually do treat the counting numbers differently from measured numbers.  So if we measure with a ruler 3.1 cm then we say that number has 2 significant figures and when you do math with it you have to count those two sig figs and no more.  

Now when we are counting that is different from measuring.  If I measure with a ruler then I can say it was real close to the 3 line, but I can never say it was exactly that.  We could always zoom in more until we have a microscope that can see the Planck length.  

But when we are counting we know exactly.  That isn't like measuring.  If I count 3 cows in the field, looking closer isn't going to reveal 0.001 cows that we just couldn't see before.  So when you're doing calculations in chemistry class and you're counting your sig figs, you actually count the counting numbers as infinite accuracy to keep them from interfering with the number of sig figs coming from the measured numbers.  So in that treatment you are pretending that the integer has an infinite number of digits.  Not that you would ever write it that way, but for the purposes of the calculation you treat it as that exact.  
If at first you don't succeed, up - home - sudo - enter.

Delta_G

Take a freshman level number theory class or just read the text and you'll understand.
I have.  And I'm talking about quantity, not number set theory.  To say that I have a certain quantity does not limit me to any particular set of numbers narrower than the set of real numbers.  Perhaps I should concede that I should have used the word quantity instead of number in the original line that set this off. 

Either way, I think we're both arguing true points about two completely different things. 

Here's another point for the OP if he is still with us.  We work with a number all the time that has a whole bunch of digits.  Think about pi.  If you are working with pi, do you need to make sure you capture ALL of the digits (an impossibility) or do you round it at some point where it is accurate enough for your particular needs?
If at first you don't succeed, up - home - sudo - enter.

jremington

Quote
For solution number 2, you gotta be kidding.
I would never kid anyone who can't be bothered to read basic documentation, and is actually surprised to learn about buffer overflow!

alirezasafdari

Honestly, I am really sad to realize that non of you try to answer the question and instead try to find off topic stuff that are not even related.

As you clearly see in the title, I have asked for a safe way. I totally understand the concerns you guys highlight with significant digits and all but that is totally unrelated to the question.

My interest is not in showing the user a huge number, my interest is in making sure if such a number for any reason was placed in the float (which is possible, even though it may be stupid or whatever) and the typical dtostrf function is called, it does not change the memory content of adjacent locations .

I seriously do not understand why so much focused is put on the meaning of the number rather than the possibility of dtosrf being called with a huge number. Also take note it can be possible that the user misses a very rare case when a number which is supposed to be one digit, become 2 digits and the buffer is only allocating enough room for one digit. the current dtostrf is going to write in another memory location which is not supposed to.

One solution is to check every float before calling the dtostrf but as I have mentioned before that would take so much time since you have to make sure each number is within the positive and negative boundary. That is 2 float comparison for each time dtostrf is called and the boundary changes
 based on formatting. I really do not consider this a solution, if you do; then you may just stick to this solution and hopefully you will have enough time for everything else in the project.

Yeah, I may have misread the documentation but that is because the dtostrf is very unsafe and I had higher expectation from the general dtostrf. This implementation is ok for when you have an operating system or something to stop you from screwing the memory but in arduino we clearly do not have such a guard and you can try all you want talking about significant digit and all the off topic stuff you have written here but you do not change the fact that this function is very unsafe and making it safe is actually very very easy. As I stated I will do it myself and share the code here withing next few days, so probably folks who understand code safety can be benefited.

After all if you have suggestion on topic, I will be more than happy to hear it.

CRAE TECH

Delta_G

The thing you don't understand is that there's no difference in how that huge number is represented and how 1.0 is represented insofar as number of digits is concerned.  They're both going to be 32bit floating point which has about 6 or 7 digits of precision.  When you call dtostrf you get to specify the precision so you get to specify how many digits get printed. 

If you're really worried about it, re-enable floating point support for format strings and use snprintf
If at first you don't succeed, up - home - sudo - enter.

alirezasafdari

Delta_G Please read the last part first.

"how that huge number is represented and how 1.0 is represented"
You are wrong!
Code: [Select]

{
float magicNumber = 1.0;
....
float a = magicNumber;
char buffer[3];
dtostrf(a, 0, 1, buffer);
}

[1][.][0]
works perfectly fine but
Code: [Select]

{
float magicNumber = 1000.0;
....
float a =magicNumber;
char buffer[3];
dtostrf(a, 0, 1, buffer);
}

[1][0][0][0][.][0]
is a disaster. I hope you understand that.

Yes this could be avoided if we could know the exact value that comes out of magicNumber. But this magicNumber may come from another unit and if that unit fails to stay in range, you have to make sure the arduino side does not explode = writing in memory location that does not belong to the buffer.


"They're both going to be 32bit floating point which has about 6 or 7 digits of precision."
I do not know how is this even related. Yes all floats and doubles take 32 bit in Arduino but does it mean when they are printed by dtostrf with zero digit after the decimal point, the take same amount of memory? definitely not. (I hope you understand this too)


"When you call dtostrf you get to specify the precision so you get to specify how many digits get printed. "
You have probably read the link below but this is not how the arduino dtostrf work. You have probably read "prec determines the number of digits after the decimal sign" but in arduino it is "prec determines the number of digits after the decimal point". You can give it a shot if you do not believe me.

http://www.atmel.com/webdoc/avrlibcreferencemanual/group__avr__stdlib_1ga060c998e77fb5fc0d3168b3ce8771d42.html

I think this is the reason why there is confusion here. I also checked the stdlib.h and the description is wrong.


I do not think the rest of the sprintf over head is not worth it nor needed.
CRAE TECH

westfw

That does seem a bit annoying, doesn't it?
How about dtostre(), which prints in the "+1.2345e-06" format?  You still get to specify "precision" for number of places after the decimal point, and that should yield a known maximum total width...


Delta_G

That's the one I was looking for.  I knew you could get it out in scientific notation.  That's what you do with big numbers.  dtostre
If at first you don't succeed, up - home - sudo - enter.

krupski

Hi

I have been using in dtosrtf() in a relatively big project and some usual stuff was happening. Long story short, I took "The caller is responsible for providing sufficient storage in s" not as serious as I should. I thought if I do not provide enough width it automatically understand not to write in the rest of the memory.

Someone in another forum suggested using 100 byte array to be safe, but I think this is not a reasonable solution on a AVR/Arduino with limited memory.
So sad that the Arduino IDE developers refuse to let users do something like this:

Code: [Select]
int main (void)
{
    init();
    Serial.begin (115200);

    STDIO.open (Serial);

    double a = 123.45678;
    double b = -0.0045;
    double c = 1.2345678;

    fprintf (stdout, "The variable 'a' has a value of %7.3f\n", a);
    fprintf (stdout, "The variable 'b' has a value of %7.3f\n", b);
    fprintf (stdout, "The variable 'b' has a value of %7.3f\n", c);

    while (1);
}



Output from this code:

The variable 'a' has a value of 123.457
The variable 'b' has a value of  -0.004
The variable 'b' has a value of   1.235

See? No "dtostrf" baloney.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Delta_G

So sad that the Arduino IDE developers refuse to let users do something like this:

Is it really the IDE that stops you?  I thought it was GCC that was the issue there.  And they don't refuse to let you.  You did it.  It's open source, you can do anything you want with it. 
If at first you don't succeed, up - home - sudo - enter.

krupski

Is it really the IDE that stops you?  I thought it was GCC that was the issue there.  And they don't refuse to let you.  You did it.  It's open source, you can do anything you want with it.  
GCC does everything just fine. All the IDE does is build GCC command strings and passes them to the compiler. Any limitations in programming an Arduino are due to the IDE, NOT GCC.

To use ordinary printf and floating point, all that's necessary is to "hack" the IDE to link  libprintf_flt.a  into the compiled binary instead of  libprintf_min.a.

Or, better yet, add a checkbox option in Preferences to choose which one to link (floating point when you need it, the non-floating point one when you don't - saves a few bytes of flash).

The "sketch" I show above was compiled by a plain old Arduino 1.0.5 IDE with a few minor differences:

(1) I turned on the floating point option checkbox (custom option in my IDE).
(2) I connected the serial port to the standard IO streams to allow using fprintf (the STDIO.open() call).
(3) I used "int main (void)" instead of that "setup/loop" nonsense (although it would work the same with setup/loop).

See? I didn't need "dtostrf". I didn't need to worry about buffer overflows. I didn't need a dozen "Serial.print()" calls just to print a few lines, I didn't need to play games checking string lengths and inserting padding to get the numbers to line up right.
Gentlemen may prefer Blondes, but Real Men prefer Redheads!

alirezasafdari

@westfw that could be a solution but honestly showing the scientific notations may not be the best way to present data mainly it is harder to read and understand if the numbers are small most of the time. So, I am still going to work on a solution which safely perform this. Please read the rest of my post after replies to each individual, because I have a question which probably you can answer it the best.

@krupski I am not really familiar with IDE manipulations and most stuff in that regard which you mentioned. I think the method you proposed has a lot of over head in run time. but a "safe, civilized, reasonable" dtostrf is really easy to do and also consider that we may not always need to put things in serial port (although I know sprintf could be used if some tweaks are done to IDE).



Now back to question:

1) so is it concluded that the description for dtostrf is wrong or dtostrf is implemented wrong? if yes, could you guys with higher star rating and reputation voice this out so that it can get fixed. Or if I should report it myself please let me know what is the best way to do it.


2) I was checking the implementation of dtostrf from this (thank to westfw) and I noticed there is an interesting function which does most of the job and probably is written in the most efficient way. that function/macro looking function is __ftoa_engine.
A good documentation can be found in here . Before finding this document I noticed that __ftoa_engine cannot be used in arduino IDE. and I could not even include ftoa_engine.h. How can I get access to this file in arduino? should not it be part of the IDE?


3) In the reference above for avr-libc.1.8.0 a dtostrf implementation has been provided which follows the exact documention as the one used in arduino but it has been implemented correctly, which means the user can tell the precision."prec determines the number of digits after the decimal sign". What do you think about this implementation?
CRAE TECH

Go Up