Go Down

Topic: Fast data logger for multiple analog pins (Read 7977 times) previous topic - next topic

Movykappa

Hello fat16lib,

Many thanks for the file. Seems to work really well. I was trying to make this fast analog reading with the analoglogger from the sdfat library, which was having quite trouble to make it below 20ms.

Let me make a suggestion for improvement, a code that reads the already present csv files on the SDcard and doesn't overwrite them!

Keep up the good work,

fat16lib

Movykappa,

Thanks for the suggested improvement.  Many people have suggested other features and "improvements".

The purpose of this examples is to provide a simple base for fast data logging that can easily be modified so I have not complicated the example with the many suggested additions.

I will likely add a function to SdFat to create a new file with rotation numbers.

The function would be used like this:
Code: [Select]

  if (!file.createUnique("LOG.CSV")) {
    Serial.println("createUnique failed");
    while (1);
  }

This would create and open a new empty file with name "LOGnn.CSV" where  nn is 00 - 99.  This would prevent writing over existing files.

wilhem

#17
Jun 26, 2013, 07:26 pm Last Edit: Jun 26, 2013, 07:29 pm by wilhem Reason: 1
Hello everybody.

I decided to create another analog datalogger for my RC aircraft but I need some help and tips for design.
Since I started learning FreeRTOS for Arduino, I realized soon, that the best way is to use an real time application for savings data on a SD card.

So here the general specifies:


  • Task 1 ("T1", high priority) reads the 6 analog pins every 4 millisecs and save them in a struct. Then move the struct into a queue;

  • Task 2 ("T2", lower priority) reads the struct from the queue and save it into a SD Card. This Task runs everytime T1 is blocked;


The biggest problem is that I don t know how to efficiently copy data from the queue to the SD card. Since it could last milliseconds, I thought 2 strategies:

  • to append during T2 the whole queue into the file and then close it => super Problem: it lasts more than 4 - 0.6 = 3.4 milliseconds;

  • to append during T2 just few values from the queue to a static buffer (which can store about 2000-3000 values) and only when the buffer is completely full then transfer all the buffer to the file and close it (during this operation I would stop T1, since I don't need more values from  the sensors).

 

I have no idea, how it is better to implement it. Since I don t have too much time between T1 und T2 I m afraid I can mess all data in the queue. Do you have a much better idea !??!?! Please…you are welcome!!!

PS: I know that there are more efficient RTOS out there, like NilRTOS for istance, but please, I m learning now FreeRTOS and I bought the application guide for microcontrollers, so I don t want to get crazy learning another RTOS.

wilhem

Hello

Ok. I've read the documentation (the examples) of NilRTOS and I understood the idea, to create a buffer, in wich data are going to be stored and read according to the FIFO rule. And both tasks are going to be synchronized to put and read data from the buffer.
maybe it is a better idea, dont know... may I ask, if my idea to put data in  the queue with FreeRTOS functions synchronizing the tasks is better than to create a single buffer like the example in NilRTOS!?!?!??

I would like to understand more....

Thank you very much

Regards

Mplex

Hello,

I tryd to use this sketch for logging with a manual on/of switch on a digital pin instead of serial character.
I tryed simply to replace " while (!Serial.available()) {  "  with    " while (!Pinbutton high()) {   "
but couldn,t get it to work.
any suggestion for this    Where to place the digitalpin read ?

klass


pito

#20
Oct 09, 2013, 10:58 pm Last Edit: Oct 09, 2013, 11:00 pm by pito Reason: 1
Quote
maybe it is a better idea, dont know... may I ask, if my idea to put data in  the queue with FreeRTOS functions synchronizing the tasks is better than to create a single buffer like the example in NilRTOS!?!?!??


I do not know how it is with Freertos, but I did it with Nilrtos - the NilFIFO.h is a ready class, and it works. I sampled and wrote struct=88bytes of data 333x per second to the sdcard without lost bytes (with 1284p and 14kB ram used).

The FIFO is needed because of the sdcard's write latencies (up to 250ms, typically 10-80ms). So you want write your struct into FIFO with the first task at sampling intervals, with the second one you write the struct from FIFO to the sdcard (when the FIFO not empty). During sdcard's outages the FIFO fills itself, when the sdcard is ready to accept the writes the FIFO empties..

Mind the FIFO ram size depends on struct size and the sampling period and expected latencies. With ie. struct=100bytes and with 200x per second sampling rate and 250ms worst case sdcard's latency the FIFO needs 100x200x0.25=5kBytes of ram.

pito

Quote
I tryd to use this sketch for logging with a manual on/of switch on a digital pin instead of serial character.
I tryed simply to replace " while (!Serial.available()) {  "  with    " while (!Pinbutton high()) {   "
but couldn,t get it to work.
any suggestion for this    Where to place the digitalpin read ?


Try:
Code: [Select]
while( !(digitalRead(buttonPin) == HIGH))

hc114

Hi!
I was wondering, would this program be able to log 100,000 samples? I have an Arduino Uno code that is able to log 100,000 samples with a high quality SD card but it can only log  one analog input. I upgraded to a Mega and I was wondering if this multiple analog input logger can go up to 100,000 samples. I only need to log maximum 3 analog inputs.
Thanks!

fat16lib

#23
Jul 07, 2014, 11:29 pm Last Edit: Jul 07, 2014, 11:37 pm by fat16lib Reason: 1
No.  

The AVR ADC can log one pin at 100,000 samples per second but accuracy is reduced to under 8-bits.  It is not possible to log faster.

It is possible to log much faster with modern ARM devices using DMA.  I have been experimenting  with STM32 chips with a 2.4 MSPS ADC.  This ADC can automatically log multiple channels using DMA at (2,400,000/<number-of-channels>). per second.

Edit: Are you sure you are not missing points at 100,000 samples per second?  How are you doing the 100,000 samples per second?

Oskarmust

Could this be done on the Arduino Yun?

Go Up