analog input - Sample and Save to SD card

Hello everyone,

I’m using a self-design Arduino board with ATmega 644PA CPU.

I’m reading 2 analog inputs and I need to sample them in the fastest rate as I can and then save it on the SD card, so far I was able to do so but with very low sampling rate.

I tried to use array and strings but it didn’t help much and I had loose data problems with both options.

what can I do to make it better?

the following code gives me a sample rate of ~25ms how can I lower it?

if I will write to array or a string and then save to SD card it still will have a 25ms delay between saves.

#include <SD.h>

const int AN0_pin = A0; int genvolt = 0;
int AN1_pin = A1; //Amp
int AN2_pin = A2; //fdbk

unsigned long time = 0;
const int chipSelect = D4;  //sd card cs
char filename[15];
String dataString = "";

void setup() {
  Serial.begin(115200);

  dataString = "";
  dataString += String("time[ms]")  ; dataString += ";";
  dataString += String("GENV")  ; dataString += ";";
  dataString += String("A1")       ; dataString += ";";
  dataString += String("A2")      ; dataString += ";\n";
  
  // sd card 1st setup
  Serial.println("Initializing SD card...");
  // see if the card is present and can be initialized:
  if (!SD.begin(chipSelect)) {
    Serial.println("Card failed, or not present");
    while (1); // don't do anything more:}
    Serial.println("card initialized.");
  } else (Serial.println("Done"));
  //End sd card 1st setup

  //open new file
  strcpy(filename, "LOG000.txt");
  for (int i = 0; i < 1000; i++)  {
    snprintf(filename, sizeof(filename), "LOG%03d.txt", i);
    if (!SD.exists(filename)) {
      break;
    }
  }
  // test new file
  File dataFile = SD.open(filename, FILE_WRITE);
  // if the file is available, write to it:
  if (dataFile) {
    dataFile.println(dataString);
    dataFile.close();
    //    Serial.println(dataString);
  }
  // if the file isn't open, pop up an error:
  else {
    Serial.println("error opening Log.txt");
  }
} //End of Setup

void loop() {
 
// Generator read
    genvolt = analogRead(AN0_pin);
    genvolt = map(genvolt, 0, 1023, 0, 55);

    File dataFile = SD.open(filename, FILE_WRITE);
    // if the file is available, write to it:
    if (dataFile) {
    dataFile.print(millis())    ; dataFile.print(";");
    dataFile.print(genvolt) ; dataFile.print(";");
    dataFile.print(analogRead(AN1_pin)); dataFile.print(";");
    dataFile.print(analogRead(AN2_pin)); dataFile.println(";");    
    dataFile.close();
    }
    // if the file isn't open, pop up an error:
    else {
      Serial.println("error opening Log.txt");
    }

}//End Of Loop

How many readings do you want to save ?

1 less from as much as possible :slight_smile:

What is the minimum acceptable period of time over which the readings must be taken ?

around 100Hz today it is 40Hz

Tactical_Robot:
around 100Hz today it is 40Hz

I can make no sense of that reply

Do you need to open and close the file on each loop iteration? Try keeping it open for more than one sample. I don't know if it actually buffers everything you write untill you close the file, but you should be able to figure that out from the documentation and source code of the SD library.
You'll probably have to close it eventually, or you might end up with a corrupted file.

If that's still not fast enough, you can attach the ADC trigger to a timer, attach an ISR to the ADC ready interrupt, and write the new measurement to one of two buffers. In the meantime, write the second buffer to the SD card. When the first buffer is full, swap the two buffers.
This will probably be overkill if you just want a sample rate of 100 Hz.

Pieter

hi Peter

PieterP:
Do you need to open and close the file on each loop iteration? Try keeping it open for more than one sample. I don't know if it actually buffers everything you write until you close the file, but you should be able to figure that out from the documentation and source code of the SD library.
You'll probably have to close it eventually, or you might end up with a corrupted file.

I tried to do so and it helped for a while but eventually, I need to close the file and it takes 25ms.

PieterP:
If that's still not fast enough, you can attach the ADC trigger to a timer, attach an ISR to the ADC ready interrupt, and write the new measurement to one of two buffers. In the meantime, write the second buffer to the SD card. When the first buffer is full, swap the two buffers.
This will probably be overkill if you just want a sample rate of 100 Hz.

the buffer is a good idea, can you point out for an example please?

UKHeliBob

I meant that I need 100 samples per second and today I have only 40

Tactical_Robot:
I tried to do so and it helped for a while but eventually, I need to close the file and it takes 25ms.

How long do you have to measure? Do the samples have to be equally spaced in time?

Tactical_Robot:
the buffer is a good idea, can you point out for an example please?

I don't have an example, but if you want to go that way, split it up into subproblems. Try reading the ADC on a timer first. Start with one ADC channel. The datasheet should have all the answers to interrupt/timer/ADC related questions.

Swapping the buffers is just keeping a counter, and swapping two pointers when it's full (i.e. the counter reaches the buffer size). Writing a buffer to a file is very similar to what you're doing already.
Communication between ISR and the main loop should happen through volatile flags. The pointers should point to volatile data, and should be volatile themselves as well (e.g. volatile uint16_t * volatile).
You'll probably also want to add a check that the main loop is done writing before you swap the buffers.

you really got me there with the volatile :slight_smile:
unfortunately, I don't have so much knowledge on Arduino or c so it a little bit tricky for me.
I will try to do it with 2 arrays with a size of 100 each and when one is done I will write to SD card
I hope the result will be satisfied.

Thank you all

You still have not quantified how many samples are to be taken and over what period

UKHeliBob:
I can make no sense of that reply

I think he means that currently he gets 40Hz and he wants 100Hz.

UKHeliBob:
You still have not quantified how many samples are to be taken and over what period

That's a very good point. Real time does not mean "really fast", it means (or used to mean) within a specified and guaranteed time.

The OP may not need 100Hz....

I don’t have a specific sample rate,
I just trying to restore the signal in the attached files in Excel using the TXT file from SD.

I think starting a 100Hz is enough but 200Hz it better.

TEK0016.JPG

3kg_dynm_T_full.JPG

Wouldn't it be much easier to just send the data to your computer over UART/USB and write it to a file there? IIRC, even an UNO's UART supports up to 2 megabaud, which is much more than what you seem to need.

I didn't mention it but the self-design Arduino will be on a small UAV for general I/O.

What is a UAV?

Writing to SD uses a software buffer of 512 bytes; data will automatically be flushed to the card after that. This will result in an occasional delay in the monitoring.

If we ignore the fact that you want to use a spreadsheet program, you can write binary data to the card. That would result in only 8 bytes per record; so the mentioned delay will only happen every 64 records A PC application can convert that binary file to a csv/txt file that can be used with the spreadsheet program.

A better approach might be to add external memory in the form of e.g. a fram module and store the binary data in there. You can, when needed, send it to the sd card as text.

sterretje:
What is a UAV?

Writing to SD uses a software buffer of 512 bytes; data will automatically be flushed to the card after that. This will result in an occasional delay in the monitoring.

If we ignore the fact that you want to use a spreadsheet program, you can write binary data to the card. That would result in only 8 bytes per record; so the mentioned delay will only happen every 64 records A PC application can convert that binary file to a csv/txt file that can be used with the spreadsheet program.

A better approach might be to add external memory in the form of e.g. a fram module and store the binary data in there. You can, when needed, send it to the sd card as text.

Unmanned Aerial Viehcal

can I somehow continue reading from the analog input while he writes to sd card?
I don't mind if I will have a delay of 25ms but not every sample as I have it right now

I taught about buffer array but I'm not sure how to do it..

A buffer is a good idea. What have you tried? If you post your attempt and specific questions about the part where you got stuck, we can work from there.

You will still have a longer delay when the buffer is full.

Also try to measure the parts that consume the most time. I presume it’s either SD.open or datafile.close. Most likely not datafile.print, but who knows.

If you use a timer to trigger the ADC as mentioned earlier (with the two buffers), you can keep on reading while it’s writing to the SD card, but I wouldn’t go that route unless it’s absolutely necessary.

Are you sure that the SD-card or its interface is even capable of handling the throughput you require? Have you tried to benchmark if writing the amount of data is at all possible? Also mind that the "print" function causes data conversion to string - this also consumes time that wil lower your loop speed.

I would log the readings to a binary format and convert the binary file to CSV on a computer. It is faster and requires less work to be done and data to be written per reading.