How to do Data logging of live Analog sensor readings

Projects General Guidance
1

i have in mind to learn about the basics of a self-balancing robot, i probably don't have accurate enough hardware but i think it should be sufficient to work through first principles and then that wil point me to what sort of specs i should be looking at for a proper self-balancing robot/vehicle.

i have an Arduino Uno, an (MMA7361) accelerometer and a (DFRobot) motor shield running 2 x 9V DC motors, max. current 0.65A
i will also add an ultrasonic ranger - i've not attempted connecting "so many gizmos" to the Uno in one go so i'm not sure whether i will start to experience processing power limitations.

anyway, i intend to add an SD card module to log the sensor data (mainly the accelerometer) as the "robot" goes about on it's merry way, up to now it's just been tethered while i see what the sensor data looks like via the Serial Monitor.

firstly, is my assumption correct, that it's better to store the data in a variable (program space ?) rather than to continually write to the SD card as it would slow down the processing speed ?

i have assumed this method and proceeded to write some code for it like so;

const int led=13;
const int axisX=A1;
const int axisY=A2;

float avgX , avgY ;
const float setcount = 50;

int count=0;
byte storeData[638][2];  // highest possible number of pairs that can be stored
// freeRAM = 515 at 638 pairs, but how to calculate the limit ?

#define DEBUG 1

void setup() {
  Serial.begin(9600);
  pinMode(led,OUTPUT);
  Serial.println(freeRam());
}


void loop()
{
  // read the input on analog pin 1 on a moving average:
  avgX = ( (avgX * ( (setcount-1) / setcount) ) + (analogRead(axisX) * ( 1/setcount) ) );

  // read the input on analog pin 2 on a moving average:
  avgY = ( (avgY * ( (setcount-1) / setcount) ) + (analogRead(axisY) * ( 1/setcount) ) );

#ifdef DEBUG
    Serial.print(avgX);
    Serial.print(",");
    Serial.println(avgY);
#endif

  int roundX = avgX;       // typecast to int or byte even ! (<255)
  int roundY = avgY;      // to save prog.space on data storage

  // valid range of analogRead-ings => min=280 to max=350
  // extended within 8-bit resolution : 190 to 440
  byte dataX = constrain (roundX, 190, 440);
  byte dataY = constrain (roundY, 190, 440);

 // store data to the array
  byte buffer[2] = {dataX,dataY};
  for (int i=0; i<2; i++)
  {
    storeData[count][i] = buffer[i];
  }


// code for the "robot" moves will go here; 

// << if falling backward, move forward >>

// << if falling forward, move backward >>

// << if balanced, stay still >>


  if (count < 638)
  {
    count += 1 ;
  }
  else
  {
    writeDataSTOP() ;  // store accumulated data and halt program
  }
  delay(10);        // delay in between reads for stability

}

// temporary placeholder for eventual function to stop the robot and save the data to SD card
void writeDataSTOP()
{
  Serial.println("limit reached, saving readings to SD card");
  while(1);
}


// copied this from a tip on checking RAM usage
int freeRam () 
{
  extern int __heap_start, *__brkval; 
  int v; 
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval); 
}

secondly, the storing of the data seems rather laborious with typecasting from float to byte (which i assume is better to be able to store more readings ?) and am wondering if there's a more efficient way to do it ?

EDIT:
just realised i still need to further subtract the constrained values to be 0-255

  byte dataX = (constrain (roundX, 190, 440) - 190);
  byte dataY = (constrain (roundY, 190, 440) - 190);
2

i've tried to clean up the code to be more readable.

does anybody have any comments now ?

3

retronet_RIMBA1ZO:
firstly, is my assumption correct, that it's better to store the data in a variable (program space ?) rather than to continually write to the SD card as it would slow down the processing speed ?

Program space (flash memory) is almost certainly not what you want, since it's read-only memory (unless you resort to extremely tortuous mechanisms to access it, which you certainly don't want to do). It's the place where your executable code is stored.

SRAM is where data is naturally stored - it's where all your variables are stored, and the heap and stack spaces. To access this you would simply declare a variable of the appropriate size. Note that you have very little SRAM so this gives you very little scope for storing logged data - it might be sufficient, at least to prove the concept, but it's unlikely to be what you want long term. Also note that SRAM is volatile (the contents are lost when the Arduino is reset or switched off).

You do have a small amount of EEPROM storage, which is non-volatile writeable memory. You need to be aware that there isn't a great deal of it and it can only be written to a finite number of times before it becomes unreliable so don't be too gung-ho about playing with it. You also have to deal with the problem of getting the logged data out of the EEPROM and delivering it to somewhere you can use it. This isn't really a good place to store logged data.

An SD card is far more appropriate for general purpose data logging since it has far greater capacity than any of the internal stores on the chip, can easily be transferred between devices and makes it much easier to access the logged data. You need to be aware that writing data to an SD card is not especially fast so you need to design your sketch so that this doesn't interfere with any critical real-time functions that your sketch is performing, and ensure that the rate of logging data doesn't exceed the rate that you can write to the SD card.

4

Hello Peter,
thanks for your response - very much appreciated ! :slight_smile:

PeterH:

retronet_RIMBA1ZO:
firstly, is my assumption correct, that it's better to store the data in a variable (program space ?) rather than to continually write to the SD card as it would slow down the processing speed ?

...
...
SRAM is where data is naturally stored - it's where all your variables are stored, and the heap and stack spaces. To access this you would simply declare a variable of the appropriate size. Note that you have very little SRAM so this gives you very little scope for storing logged data - it might be sufficient, at least to prove the concept, but it's unlikely to be what you want long term. Also note that SRAM is volatile (the contents are lost when the Arduino is reset or switched off).

i'm still learning the correct terms for all the various memory locations, yes - i was indeed referring to SRAM - which is what the freeram() was for - to tell me how much data i could store as a variable (in SRAM) given the limited amount.

so, to prove said concept - would my above laborious typecasting be the necessary way ?

PeterH:
An SD card is far more appropriate for general purpose data logging since it has far greater capacity than any of the internal stores on the chip, can easily be transferred between devices and makes it much easier to access the logged data. You need to be aware that writing data to an SD card is not especially fast so you need to design your sketch so that this doesn't interfere with any critical real-time functions that your sketch is performing, and ensure that the rate of logging data doesn't exceed the rate that you can write to the SD card.

if one is writing to the SD card, then the typecasting isn't necessary anymore, right ?
one just has to slow down the data stream to let the SD card writing function do it's work ?

5

An SD card can handle up to 20KB/s data rate with arduino's real SPI pins, while the rest of your code will hang. You will have to find the balance between time to log data and time to sense them and respond to them.

6

okay, so i have to slow down things to write to SD.

but how about using the SRAM method, just to see how that works, am i doing it correctly ?

just thought of another method - i could also stream the data through Bluetooth to a PC for recording via Serial Monitor, right ?

7

I glanced at your storeData code. Seems alright.

I thought about the bluetooth way too. It will also stall your program if you keep sending. The serial.print() will stall until there are enough bytes in the out buffer to fit your printouts. However, increasing the out buffer size will likely solve the problem.

I assume you don't exceed 115200 bits per second rate on sending data (have some timers to send say every 10ms instead of as fast as possible).

If the serial out buffer is too small:

Each byte to serial hardware takes very little time (in the "background", although one of the senior members won't like this figurative expression).

serial.print-> place printout in out buffer
do your balancing thing
serial.print-> place printout in out buffer
do your balancing thing
serial.print-> place printout in out buffer, oops out buffer full, stall until it gets empty enough to store the printout
...
...
do your balancing thing

If your buffer is large enough, then there won't be stall.

8

liudr:
I glanced at your storeData code. Seems alright.

okay, thanks - i guess i'll discover more efficient ways as i learn more about programming with more advanced tools.

liudr:
I thought about the bluetooth way too. It will also stall your program if you keep sending. The serial.print() will stall until there are enough bytes in the out buffer to fit your printouts. However, increasing the out buffer size will likely solve the problem.

oh boy, i'll have to dive deeper into the actual workings of the Serial.print function !

liudr:
I assume you don't exceed 115200 bits per second rate on sending data (have some timers to send say every 10ms instead of as fast as possible).

and this would be using "BlinkWIthoutDelay" and not delay() right ? :slight_smile:

liudr:
If the serial out buffer is too small:

Each byte to serial hardware takes very little time (in the "background", although one of the senior members won't like this figurative expression).

serial.print-> place printout in out buffer
do your balancing thing
serial.print-> place printout in out buffer
do your balancing thing
serial.print-> place printout in out buffer, oops out buffer full, stall until it gets empty enough to store the printout
...
...
do your balancing thing

If your buffer is large enough, then there won't be stall.

thanks, that makes more sense to a newbie - not sure why pseudo-code would be frowned upon.
will have to do some searching on Serial buffer now.
thanks for the tips !