next topic »

[on:]

I've been working on using the Arduino as a data logger for a while now, and I need to log data from various sensors and want to do it as fast as possible so I don't have to revert to using an NI DAQ and LabView all the time. I really need to log data very quickly and am having some issues getting above 200 Hz, so I thought I would simplify things today and get back to basics to try to find the maximum speed I can write to a file. The first thing I tried was just using a photocell hooked up like one of the FSRs http://itp.nyu.edu/physcomp/Labs/AnalogIn, then I tried a speed test to see just how fast I could get data to print. While you can see data scroll in the serial monitor screen of Arduino, you can't save it to a file directly from there. I've found that CoolTerm is the easiest way to do this, and it's available in Mac, Windows, and Linux versions (yes, I'm still a PC): http://freeware.the-meiers.org/

So here's the initial code:

Code:

const int analogInPin = A0;  // Analog input pin that the photocell is attached to
int sensorValue = 0;        // value read from the photocell

void setup() {
  // initialize serial communications at 115200 bps:
  Serial.begin(115200);
}

void loop() {
  // read the analog in value:
  sensorValue = analogRead(analogInPin);           
 
  // print the results to the serial monitor:
  Serial.print(millis());   
    Serial.print(",");   
  Serial.println(sensorValue);                       
}

And here's a section of the initial output:
953,828
954,828
954,827
955,828
956,828
957,828
957,827
958,828

The millis() function outputs milliseconds of time elapsed, and as you can see above, I'm getting some readings at the same ms! Which means this is printing faster than 1000Hz. So, is there a way to get more resolution in the time function so each time stamp is associated with only one sensor reading? Turns out there is! The micros() function outputs in, you guessed it, microseconds- sort of.  According to the reference:

On 16 MHz Arduino boards (e.g. Duemilanove and Nano), this function has a resolution of four microseconds (i.e. the value returned is always a multiple of four).

So then I checked the Arduino Uno hardware page (the version I'm using) to check if it is a 16Mhz version, and it is indeed. So, lets give this another shot and replace millis() with micros() in the above code.

1064,824
1908,824
2776,824
3664,824

Bingo! Now this time is going by so fast that two sensor readings never get logged to the same time reading.  Okay, now to make sense of the data given the resolution...
I was confused by the resolution part of the statement above, but a user on the Arduino forum post I made cleared that up:

All it means is that, in effect, instead of incrementing a variable by one every microsecond, the variable is incremented by four every four microseconds.

So, there are 1,000 microseconds in 1 millisecond, and 1,000 milliseconds in 1 second, so there are 1,000,000 microseconds in 1 second.  So all I have to do is divide each reading by 1,000,000 to get seconds.  For example, the first time reading I get is 1024 microseconds = 1024/1000000 seconds = 0.00102400 seconds.  If I do that calculation for the first few data points, I get a delay of about 0.000888 seconds between readings, which means I'm logging data at about 1150 Hz.  Not bad, but not great, given that analogRead() happens at at the rate of about 10,000 Hz. So can it get better?

One possible thing slowing down the write is using 3 different commands to print the microseconds, comma, and sensor. I remember someone telling me to write data out all at once in a string. I had a little trouble figuring out how to make a string of a number, a comma, and another number at first, but finally the code below worked:

Code:

const int analogInPin = A0;  // Analog input pin that the photocell is attached to
int sensorValue = 0;        // value read from the photocell
String toprint;

void setup() {
  // initialize serial communications at 115200 bps:
  Serial.begin(115200);
}

void loop() {
  // read the analog in value:
  sensorValue = analogRead(analogInPin);
  toprint = 0;  // resets toprint to 0 each loop, instead of adding terms on to the end each time
  toprint += micros(); 
  toprint += ',';
  toprint += sensorValue;

  // print the results to the serial monitor:
  Serial.println(toprint);   
}

However, the frequency of data writing actually <strong>decreased </strong><em>to about 885 Hz.   So, is there any way I can get faster than the 1150 Hz number?

Thanks!!

[Reply #1 on:]

I don't Serial is what is slowing you down.

115,200kilobits per second means you can print 14,000 bytes (8 bit) values per second.  Or, one byte (ascii character) every 72µs.

Try printing the micros() value itself, instead of the slow String operations you are doing now.

[Reply #2 on:]

Thanks.  I tried printing the micros() value separately at first and got 1150 Hz, then tried it in the string and it went down to 885 Hz. So I'm not sure I follow you?

[Reply #3 on:]

Hi Dustyn-

    Gluing strings together ("concatenation") tends use significant clock cycles, because there is a lot of moving data around to create the string in memory, whereas when you print directly, the data simply goes into a buffer and gets sent out directly.

    One thing that might help is knowing how long you need to sample the data for? 10 seconds? 60 seconds? 5 minutes? You may be able to queue up the sensor data in ram, and then just dump it all at the end. Also, do you need to see specific values, or just large spikes?

[Reply #4 on:]

    Gluing strings together ("concatenation") tends use significant clock cycles, because there is a lot of moving data around to create the string in memory, whereas when you print directly, the data simply goes into a buffer and gets sent out directly.

I think her point (and I missed it originally) is that the first code example didn't use concat.  Using concat only cost a 100-200Hz of time.  It looks like Serial is taking longer than expected (or desired).

[Reply #5 on:]

try using Serial.write() instead since it is simply called by Serial.print(), print is just much more flexible about intput, and include Serial.write(13);
Serial.write(10);

for your carraige return and newline commands.

i have no idea how processor intensive the string class is as well, so I would loose that.

[Reply #6 on:]

since I am always asking questions and never contributing, i decided to go all out.  Keep in mind, i am a pretty bad proggrammer, and I have not tested this.

EDIT: EVERYTHING I SAID IS A LIE IN MY LAST POST

Code:

const int analogInPin = A0;  // Analog input pin that the photocell is attached to
int sensorValue = 0;        // value read from the photocell

void setup() {
  // initialize serial communications at 115200 bps:
  Serial.begin(115200);
}

void loop() {
  // read the analog in value:
  sensorValue = analogRead(analogInPin);

  // print the results to the serial monitor:
  Serial.print(micros());     
  Serial.write(',');
  Serial.println(sensorValue);

}

[Reply #7 on:]

since I am always asking questions and never contributing, i decided to go all out.  Keep in mind, i am a pretty bad proggrammer, and I have not tested this.

EDIT: EVERYTHING I SAID IS A LIE IN MY LAST POST

So in your "last post" you said use Serial.write().  In your next post you only used Serial.print(), except for the comma.  What is the point you were trying to make?

[Reply #8 on:]

Serial.write won't work the way I thought, so to only tip i had to give was get rid of the string.  However I just did some back of the envelope math, the transfer rate will not let you send more than 1 sample every 868 us, or 1150 Hz.  that is because of the baud rate, assuming you let it run for 10 seconds.  you can only send data out that fast.  if you ditch the micros() you can win back a ton of speed but loose your time stamp.  other option is to use modu to shorten micros() to something useable.

I think the only way to speed this up is to read for a certain amount of time, then spit all hte data out at once later.  how long of a sample do you need?