I have a piece of code that performs some math and prints the data to the serial port as follows.
// A Long variable comes from a function.
long tempdata1 = function(Data1);
long tempdata2 = function(Data2);
// convert the data to a real world value, here capacitance
CapCh1 = (((float) tempdata1) / 524288 ) + CapOffset - 16;
CapCh2 = (((float) tempdata2) / 524288 ) + CapOffset - 16;
// build a string and set the string out over the serial port
dataOutput = String(String(TimeStamp,6) + "\t" + String(CapCh1,6) + "\t" + String(CapCh2,6));
Serial.println(dataOutput);
I am looking for any suggestions on how to speed up this portion of the code. I would like to keep my data in the serial port in a float format, but how that happens inside the Arduino is not important. Here is a sample of the code that is returned over the serial port in the form of [time, data 1, data 2].
It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. Just use cstrings - char arrays terminated with 0.
Why not do the maths as integers and fake the decimal position in the output?
I routinely run this code for minutes (maybe 10 min max) at a 50 Hz output speed without any real issues at related to stalling. However, I am always looking for ways to improve the code.
The code is long and somewhat complicated in relation to the hardware. As my problem was simply to the code format I thought it best to just provide the simplest problem demonstration.
tex_downey:
First, the magic number 524288 is needed to convert the values returned by the I2C measurement chip to a capacitance value.
Yes. But how is it derived? Maybe you don't need to do the calculation in one big lump?
For integer math, I have an issue with the level of precision I need. My
tempdata1
has a typical value around 10,000,000 so if I want to return a value with at least 4 digits of precision I would need to multiply
tempdata1
by 10000. The resulting number would be outside the unsigned long format capability.
I you only need 4 digits of precision can't you divide 10 million by 1000 and still have 6 digits? And dividing by 1024 would be much quicker for the Arduino (power of 2).
Don't be confused. My main points were that it is possible to fill the output buffer (and as a result your code will slow down) and that you did not provide context (e.g. baudrate, how often your snippet is called and so on).
Just to explain the stalling
At 50Hz, you have 20ms to get your data out. At 9600 baud, one byte takes approx 1ms. Your data example contains 28 bytes plus 2; that requires 30 ms (at 9600 baud). So after two or three calls, your code will slow down.
Increasing the baudrate to e.g. 19200 will solve the issue in this case (you now pump the data out at 0.5ms per byte so 15ms for a full 'record'). Obviously you can even go faster.
Note
You don't have to provide complete code; just a working example that exhibits the behaviour.
tex_downey:
I have a piece of code that performs some math and prints the data to the serial port as follows.
// A Long variable comes from a function.
long tempdata1 = function(Data1);
long tempdata2 = function(Data2);
// convert the data to a real world value, here capacitance
CapCh1 = (((float) tempdata1) / 524288 ) + CapOffset - 16;
CapCh2 = (((float) tempdata2) / 524288 ) + CapOffset - 16;
// build a string and set the string out over the serial port
dataOutput = String(String(TimeStamp,6) + "\t" + String(CapCh1,6) + "\t" + String(CapCh2,6));
Serial.println(dataOutput);
I am looking for any suggestions on how to speed up this portion of the code. I would like to keep my data in the serial port in a float format, but how that happens inside the Arduino is not important. Here is a sample of the code that is returned over the serial port in the form of [time, data 1, data 2].
0.001708 92.922028 89.738556
Thanks in advance!
Austin.
Do you even need to use float?
I mean, if you are testing capacitor values, I would think that if a 100 uF cap reads as 92uF, that's way good enough. Printing a value with several decimal places is useless and pointless.
(Kinda like the wet-behind-the-ears fresh EE grads that calculate the resistor to use for an LED and then try to find a 121.372 ohm resistor!)
I misspoke. I need at least 5 digits after the decimal point. I have tried to divide the numbers beforehand without using any floats but I am not getting any increase in speed. Here is a sample code that you should be able to run.
// define variables
long tempdata1;
int CapOffset;
unsigned long SampleTime;
int TimeStamp;
int Timestep;
String dataOutput;
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
CapOffset = 90;
tempdata1 = 9913409;
}
void loop() {
SampleTime = micros(); // gets the arduino time in microseconds.
Timestep = SampleTime - TimeStamp;
TimeStamp = SampleTime;
// original code 424 uS
float CapCh1;
CapCh1 = (((float) tempdata1) / 524288 ) + CapOffset - 16;
// all interger based code 424 uS
//long CapCh1;
//CapCh1 = (tempdata1 * 10000L / 512) + CapOffset - 16; // was 524288
//dataOutput = String(String(Timestep) + "\t" + String(CapCh1));
//Serial.println(dataOutput);
Serial.println(Timestep);
}
On both the integer code and the float code I get a cycle type of 424 uS. Maybe this is just as fast as it is going to get.
If you need precision then float is a problem. You only get 6 total digits of precision with float. That includes digits before the decimal as well as after. With unsigned long you get 9 digits. If that's not big enough then maybe you need long long. If it doesn't fit in 64 bits then you've got bigger issues.
I understand your concerns, however, I do need that level of precision in my measurements. It is not a meaningless requirement, rather it is driven by the sensor that I am measuring.
I will keep trying to see if I can obtain a faster calculation with integer math.
tex_downey:
I misspoke. I need at least 5 digits after the decimal point.
Robin2:
IMHO that is a meaningless requirement.
The question is, and remains, how many total digits of precision do you need? Knowing how many on the right side of the decimal doesn't tell the whole story. Precision is measured by the total number of significant figures so you have to also know the number of digits needed on the left side of the decimal. Or maybe there are none on the left side that matter in which case you ask how many of the ones on the right will be known. If you needed ten digits behind the decimal, but no measurement was more than 0.0000000999 then you only need 3 digits for the math. Or if you needed 4 places behind the decimal but all the measurements were between 14564.923 and 14564.988 then you only really need two digits of precision and you can make the math work.
If you're measuring in meters and need a bunch of decimals, measure instead in micrometers and do it with integers. The math is way faster that way.
See the problem with floats is that while they can handle 0.00000000345 very accurately, if you tried to hold 1234.567890 then those last 4 digits won't be exact as it comes through the math. And with every operation you do, that error adds up.
Try using the pow() function to find the powers of 2 and store them in an int. See what comes out. That's the weakness of floating point math.
void setup(){
Serial.begin(115200);
for (int i = 0; i < 32; i++){
int num = pow(2, i);
Serial.println(num);
}
}
void loop(){}
Notice something odd? Shouldn't the powers of 2 all be even numbers? It's because pow returns a float.
DISCLAIMER: I didn't test or compile this code, but I've seen the issue come up several times on the forum. I know it happens.
Thanks for taking the time to explain the issues with float based math to me. Here are a few comments,
Accuracy is more important than speed for me, so I have a great intrest in getting this code right.
I am measuring strain (material deformation) with a capacitance based sensor. The changes in capacitance for a change in strain is fairly small, this is why I have defined my need for precision as the number of digits to the right of the decimal point. For example, a 0.00001 pF change in capacitance relates to 0.4 microstrain. Any more precision is well beyond the sensor's ability. This is why I need those final digits to come through.
My sensors should always be below 100 pF, therefore, I will need 7 digits of precision (two to the left, 5 to the right). From your last comment if I change the tempdata1 in this line of code
from a long to an unsigned long I should be able to obtain the needed precision, correct? Can I leave CapOffset as a simple integer?
This project fits into a measurement ecosystem that uses pF for everything. While I could change the units at this point the simplicity and redundancy of leaving it in pF outweighs the small increase in speed. I could always do all the math in LabView or in post processing to get the maximum speed but I like having the simplicity doing the math in the microprocessors.
Accuracy is more important than speed for me, so I have a great intrest in getting this code right.
I am measuring strain (material deformation) with a capacitance based sensor. The changes in capacitance for a change in strain is fairly small, this is why I have defined my need for precision as the number of digits to the right of the decimal point. For example, a 0.00001 pF change in capacitance relates to 0.4 microstrain. Any more precision is well beyond the sensor's ability. This is why I need those final digits to come through.
Now we are getting a little closer to the root of the issue. You can measure changes of 0.00001 pF. But we also need to know the range between the max and min pF values that can occur. ONLY then can we know what level of error is acceptable. For example if the range is from 0.01000 pF to 0.00001 pF then you are looking at 1 part in 1000.
A range of 1 to 1000 microstrain would relate to a capacitance from 0.00001 to 0.01 pF, or 1 part in 1000. These would be typical values for a typical installation.
I find it hard to believe that a capacitance meter exists that has a range of 0-100pF and an accuracy of 0.0001pF. Potentially it could have a resolution of 0.0001, but that's not the same.
I looked at your website and seen the graph, which shows a change from 0 to about 7pF when something happens (I know nothing about strain gauges so can't comment further on the 'something!). But there is also a lot of noise around the 0pF level. Assuming that always exists, surely you don't need accuracy to .0001pF? Or, if I'm wrong about the 'noise', and if you're more interested in a relative change, from say 0.0002pf to 0.0005pF, do you really need to also measure up to 100pF?
tex_downey:
I misspoke. I need at least 5 digits after the decimal point. I have tried to divide the numbers beforehand without using any floats but I am not getting any increase in speed. Here is a sample code that you should be able to run.
// define variables
long tempdata1;
int CapOffset;
unsigned long SampleTime;
int TimeStamp;
int Timestep;
String dataOutput;
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
CapOffset = 90;
tempdata1 = 9913409;
}
void loop() {
SampleTime = micros(); // gets the arduino time in microseconds.
Timestep = SampleTime - TimeStamp;
TimeStamp = SampleTime;
// original code 424 uS
float CapCh1;
CapCh1 = (((float) tempdata1) / 524288 ) + CapOffset - 16;
// all interger based code 424 uS
//long CapCh1;
//CapCh1 = (tempdata1 * 10000L / 512) + CapOffset - 16; // was 524288
On both the integer code and the float code I get a cycle type of 424 uS. Maybe this is just as fast as it is going to get.
OK, 424 microseconds. The time it takes to Serial.print about 4 or 5 characters at 115200 baud. How many characters (including terminators) are printed each cycle? Coincidence or ...?
The truth is that the only thing that you are timing there is the serial print. With all of the numerical inputs fully known at compile time, the compiler has completely optimized every single calculation out of your loop.
tex_downey:
3. My sensors should always be below 100 pF, therefore, I will need 7 digits of precision (two to the left, 5 to the right).
No, you still don't understand. If your measurements are all between 1 and 100 pF you only need 3 digits of precision. You just do the math in pF instead of F so you can use integers.
