Go Down

Topic: HC-SR04: tests on accuracy, precision and resolution of ultrasonic measurement (Read 22315 times) previous topic - next topic

RiGonz

Hi to all!
I have been using the ultrasonic sensor HC-SR04 for a while.
I was puzzled to read in most pages that people were happy with them because my experience was mixed.
Now I am planning to use them for a project where I need to know more about their specifications, so I have run some domestic tests on their accuracy, precision, resolution, and angular and linear ranges.
I have uploaded a document with the procedures and results here:
https://app.box.com/s/sj7du1n32in2777rcoi2
The summary is:
"
Based on my tests the specifications of the HC-SR04 are:
-   FOV (full cone): horizontal ~21º, vertical ~4º
-   Spatial resolution (full cone): ~0.6-1.4º
-   Range: tested from 5 to 200 cm
-   Accuracy: absolute error ~0.035 cm/cm.
-   Precision: standard deviation ~0.1-0.5 cm
The use of these sensors needs attention in the interpretation of their readings. Specific testing is recommended, as well as repeated readings and filtering of the data prior to any calculation based on them.
"
Hope this helps!



Hackscribble

Hello RiGonz

This looks like a thorough and useful piece of work.  Thank you for doing it.

A couple of questions:

1.  From the pictures, it looks like the testing was carried out at floor level on a hard floor surface.  Do you think this had any effect on the low vertical FOV you measured? 

2.  Could you post the sketch you used?  It would be useful for anyone wanting to repeat the tests, or develop new ones, to be able to do so consistently with your approach.

All the best

Ray
Hackscribble.  Writing about making things.
arduino@hackscribble.com | www.hackscribble.com

RiGonz

Hi, Ray & Thanks for the feedback!

I have uploaded the sketch (pretty simple, in any case) and one of the spreadsheets that I have used to handle the data of each measurement (all are the same, only the data changes). Links (code for the sketch is also included below):
https://app.box.com/s/uez1df7dm10bfd69m1qr (spreadsheet) and https://app.box.com/s/ime2dmi9vbypyqf58qpo (sketch).

I found quite strange the very low vertical FOV that I measured, but the procedure seems simple and the results quite clear: the elevated object was only spotted farther than from a certain point, not closer.
I noticed your point on the reflected sound echos on/from the floor while reviewing the data, but the measurements didn't show any noticeable change in the absolute error or the standard deviation. If reflection was a real problem I would had expected mixed measurements, which didn't show up in the records.
Nevertheless, I think it may be worthy to test the VFOV turning the sensor 90º, just to be sure that these reflections are not any problem. Will do it and post the results, probably this coming weekend.

Rgds,

Code: [Select]
/*
// Presentation
//
* Trials with Ultrasonic sensors
* R01: Two US sensors HC-SR04
* R02: One LCD Serial
* R03: One US sensor; the reading is done directly, without <Ultrasonic.h>
* Source of sensor interface code: http://www.arduino.cc/en/Tutorial/Ping
* Control of max/min distance range also introduced
* R04: Two US sensors - not tyet crossed for reading...
* Temperature correction for air speed
* R05: Testing of the US sensors
* Only one sensor at the time, output by serial display (PC, no LCD), capture of readings by CoolTerm
* Arduino board: mini Pro
* Number of readings and time interval can be controlled
* No control of distance range
* R05b:
* The number of measurement repetitions, NR, is now better coded
*/

// Definitions
//
// US sensor pin definitions:
int pingPin = 9; // trigger pin in the US sensor
int pongPin = 8; // data pin in the US sensor

// Other variables
#define NR 10 // number of repetition of readings under each test condition
long duration; // microseconds
float tempAir; // Celsius
float soundSpeed; // m/s
int AT[5]={1,10,50,100,200}; // time interval between consecutive readings, milliseconds
float distance[NR]; // array with the readings of the repeated measurements, cm

// Setup
//
void setup()
{
  Serial.begin(9600);

  tempAir = 20.;
  soundSpeed = 331.3+0.06*tempAir; // m/s
  soundSpeed = soundSpeed/10000.; // cm per microsec
 
  pinMode(pingPin, OUTPUT);
  pinMode(pongPin, INPUT);

}

// Main Loop
//
void loop()
{
  for (int i=0; i<5; i++) // for each AT interval
  {
    for (int j=0; j<NR; j++) // for each of the NR repetitions
    {
    duration = durCalc(pingPin, pongPin); // measures the sound travelling distance (microseconds)
    distance[j] = duration*soundSpeed/2.; // converts the time into a distance (cm)
    delay(AT[i]); // wait as required between consecutive readings
    }
   
    // print results as a batch:
    Serial.println(" New Batch of Readings");
    Serial.print(" NR = ");
    Serial.print(NR);
    Serial.print(" AT = ");
    Serial.print(AT[i]);
    Serial.println(" milli seconds ");
    for (int j=0; j<NR; j++)
    {
      Serial.print(" reading[ ");
      Serial.print(j);
      Serial.print("]= ");
      Serial.println(distance[j]);
    }
 
  // ready for next loop:
  delay(1000);
  }
}

// Functions
//
float durCalc(int pinI, int pinO)
{
  // The PING is triggered by a HIGH pulse of 10 or more microseconds.
  // Give a short LOW pulse beforehand to ensure a clean HIGH pulse:
  digitalWrite(pinI, LOW);
  delayMicroseconds(2);
  digitalWrite(pinI, HIGH);
  delayMicroseconds(12);
  digitalWrite(pinI, LOW);
  //delay(10);
  // The pongPin is used to read the signal from the PING))): a HIGH
  // pulse whose duration is the time (in microseconds) from the sending
  // of the ping to the reception of its echo off of an object.
  return(pulseIn(pinO, HIGH));
}

teckel

Having spent a lot of time on this myself, I would suggest the following...

First, your calculation of the speed of sound is incorrect and you probably shouldn't be dividing by 10,000 for your speed of sound variable.  The reason is that float is only accurate to 6-7 decimal digits of precision and by dividing by 10,000 you're putting a lot of undue zeros in your variable (thus possibly reducing accuracy).  Instead, use the following:

Code: [Select]

soundSpeed = 331.3 + (0.606 * tempAir); //  m/s


Then, for your actual distance calculation:

Code: [Select]

distance[j] = duration / 20000.0 * soundSpeed; // converts the time into a distance (cm)


I would also suggest making the last line:

Code: [Select]

return(pulseIn(pinO, HIGH, 35000));


So it doesn't wait for a full second if no "pong" is heard (also prevents echo's generating false results).

Further, I question your generic "20.0" for the temperature.  While minimal, an exact reading should be taken for accurate results.

Personally, I would also rework the code so it doesn't use any floats.  This would reduce the compiled code size, speed up the program, and therefore increase accuracy.  It would be quite easy to do and still give you the same precision of results.  However, I'd doubt it would make any measurable difference so it would probably be a useless exercise, it's just how I would do it for accuracy's sake (and I avoid floats at all cost).

Tim
My hardware: Arduino, Teensy 2.0, Teensy 3.2 & custom ATmega328
My libraries: NewPing, LCDBitmap, toneAC, toneAC2, NewTone, TimerFreeTone
My Projects: https://dogblocker.com & https://baconorbeer.com
My beer: Great Lakes Brewing Co. Lake Erie Monster

RiGonz

Thanks a lot, Tim - I promise that I'll never ever write again that a sketch is "pretty simple"...
:)
The typo in the sound speed expression is quite probably the cause of the large absolute errors that I "observed".
Regarding the room temperature, I checked it and was 23-24ºC - as you mention, not a big difference, but if I added the possibility I should have made use of it...
And I will look at the possible numerical errors related to using too many floats!
Hope to have it all done by this weekend!
:smiley-eek:

teckel


Thanks a lot, Tim - I promise that I'll never ever write again that a sketch is "pretty simple"...
:)
The typo in the sound speed expression is quite probably the cause of the large absolute errors that I "observed".
Regarding the room temperature, I checked it and was 23-24ºC - as you mention, not a big difference, but if I added the possibility I should have made use of it...
And I will look at the possible numerical errors related to using too many floats!
Hope to have it all done by this weekend!
:smiley-eek:


I expect you will see very little difference.  But, a few slight inaccuracies could add up to a *slight* measurable difference.  Not dividing by 10,000 till the ping alone will fix the numerical errors (as shown in my previous post).  Converting everything over to not using floats will make things just run faster and use less program space, it won't make much of a numerical error difference.

Anxious to see if there's a measurable difference...

Tim
My hardware: Arduino, Teensy 2.0, Teensy 3.2 & custom ATmega328
My libraries: NewPing, LCDBitmap, toneAC, toneAC2, NewTone, TimerFreeTone
My Projects: https://dogblocker.com & https://baconorbeer.com
My beer: Great Lakes Brewing Co. Lake Erie Monster

RiGonz

Hi again on the tests for the HC-SR04...

I have fixed the errors spotted by Tim (thanks once again) in the sketch and even managed to work it all without float variables ( :smiley-eek-blue:); and redone the tests with the assistance of my youngest daughter :%

As per my measurements the facts regarding the HC-SR04 are:
-   FOV (full cone): horizontal ~20º, vertical ~13º
-   Spatial resolution (full cone): ~0.6-1.4º
-   Range: tested from 5 to 200 cm
-   Accuracy: relative error ~0/5% (-1.3 ±4.6 %); absolute error ~-0.5/-1.5 cm (-0.4 ±1.2 cm).
-   Precision: standard deviation ~0.1/0.5 cm (0.3 ±0.6 cm)
Worse in all regards than the standard specifications.

The related files are uploaded:
- test report: https://app.box.com/s/4nafz9scusuk4kq1628u
- spreadsheets: https://app.box.com/s/239kqs0sichzqsy63ujs, https://app.box.com/s/cq4wo2nef9ajfwdgki4b, https://app.box.com/s/4kpkrjmyh85169fr4q72
- sketch (also copied below): https://app.box.com/s/87bnz47ld3iic33mnk1u

I have worked on the results and included some charts. This one, f.eg., shows that using the mode as an estimator of the distance is not much better than the average once outlliers are removed (which is quite easier to do than calculating the mode): https://app.box.com/s/r4nwbs2qnznxhw9614of (Sorry - I still do not know how to post an image...)

Once the physical features of the sensor are reasonably clear, the issue is how to best use the readings. There are some suggestions in the forums/fora regarding this matter which, of course, recommend repeating the readings, but also using the mode instead of the average. I rather disagree with the latter: calculating the mode is computationally expensive and the average is very simple to compute and unskewed once the data is filtered. Here is my alternative proposal for "on the flight" calculations:

  • set a predefined maximum estimate of the deviation (f.eg. 1.5 cm or 3% of the value of the measurement, or both);

  • read N times the distance (suggested N= 20, or at least 10);

  • on the flight, calculate the summation of the readings (summation ? summation + reading), the main range (max, min) and the second largest range (max* = second largest reading, min* = second lowest reading);

  • once the N readings are completed:
    -   if the largest range (max-min) does not exceed the predefined deviation, then provide the average as an estimator of the distance: distance_estimate ? summation/N;
    -   else:
       if the second largest range (max*, min*) still exceeds the predefined deviation, then the reading/measurement is incorrect and therefore cancelled (no reading is returned);
       else, remove max and min from the readings sample and provide the resulting average as the estimator of the distance: distance_estimate ? (summation-max-min)/(N-2).

   
Regards,

Code: [Select]
/*
* Trials with Ultrasonic sensors
* R01: Two US sensors HC-SR04
* R02: One LCD Serial
* R03: One US sensor; the reading is done directly, without <Ultrasonic.h>
* Source of sensor interface code: http://www.arduino.cc/en/Tutorial/Ping
* Control of max/min distance range also introduced
* R04: Two US sensors - not tyet crossed for reading...
* Temperature correction for air speed
* R05: Testing of the US sensors
* Only one sensor at the time, output by serial display (PC, no LCD), capture of readings by CoolTerm
* Arduino board: mini Pro
* Number of readings and time interval can be controlled
* No control of distance range
* R06:
* Typo on sound speed ... corrected(!)
* Number of time intervals is reduced to 1 and 100 milliseconds
* All variabls are now int/long...! :-}
*/

//
// Definitions
//
// US sensor pin definitions:
int pingPin = 9; // trigger pin in the US sensor
int pongPin = 8; // data pin in the US sensor

// Other variables
#define NR 10 // number of repetition of readings under each test condition
#define tempAir 24 // Celsius - of course, can/should be changed!

long unsigned soundSpeed; // approx 340000 mm/s -> long is ok
long unsigned distance[NR]; // array with the readings of the repeated measurements, mm; precision of the readings is not beyond mm, so further digits are just false precision;
                           // long because of the operations involved.
int AT[2]={1,100}; // time interval between consecutive readings, milliseconds

//
// Setup
//
void setup()
{
 Serial.begin(9600);

 soundSpeed = 331300+606*tempAir; // mm/s; source: wikipedia
 
 pinMode(pingPin, OUTPUT);
 pinMode(pongPin, INPUT);

}

//
// Loop
//
void loop()
{
 for (int i=0; i<2; i++) // repetition with different reading intervals
 {
   for (int j=0; j<NR; j++)
   {
     distance[j] = soundSpeed/100; // speed is in mm/s, duration in microseconds: 6 zeroes need to be removed, 2 go out here, the bañance later
     distance[j] = distance[j]*durCalc(pingPin, pongPin); // distance (mm) = time (total sound travelling distance, microseconds) * speed (mm/s)
     distance[j] = distance[j]/20000; // I remove the remaing 4 zeros here, plus the effect of two-way travel of the sound; this care with the zeros is because of size limitations of long
   }
   
   // print results as a batch:
   Serial.print(" NR = ");
   Serial.println(NR);
   Serial.print(" AT = ");
   Serial.print(AT[i]);
   Serial.println(" milli seconds ");
   for (int j=0; j<NR; j++)
   {
     Serial.print(" reading[ ");
     Serial.print(j);
     Serial.print("]= ");
     Serial.println(distance[j]);
   }

   // ready for next loop:
   delay(250);
 }
}

//
// Functions
//
unsigned long durCalc(int pinI, int pinO)
{
 // The PING is triggered by a HIGH pulse of 10 or more microseconds.
 // Give a short LOW pulse beforehand to ensure a clean HIGH pulse:
 digitalWrite(pinI, LOW);
 delayMicroseconds(2);
 digitalWrite(pinI, HIGH);
 delayMicroseconds(12);
 digitalWrite(pinI, LOW);
 //delay(10);
 // The pongPin is used to read the signal from the PING))): a HIGH
 // pulse whose duration is the time (in microseconds) from the sending
 // of the ping to the reception of its echo off of an object.
 return(pulseIn(pinO, HIGH, 35000)); // microseconds of (total) sound travel;
}



vglfr

Briefly tested HC-SR04 before reading RiGonz comprehensive review.
Same conclusion: precision is not near enough of datasheet.
Test distance near 25 cm results in min-1449 max-1483 milliseconds response with 1459 median. Which means 2.5% precision. Typical measurement pattern is like 1456-1456-1455-1480-1455-1450-1456.
Shorter distance results in performance improvement. Nearly consistent results at response time 200 and below (3-4 cm).
Measurements seemed to be unaffected by sensor pulse length settings, interval between measurements, sensor and screen clamp fixation, different textures of reflecting surface.

terryking228

Hi, I added some of this information to the ArduinoInfo.Info page  HERE:
Regards, Terry King terry@yourduino.com  - Check great prices, devices and Arduino-related boards at http://YourDuino.com
HOW-TO: http://ArduinoInfo.Info

Go Up