Pages: 1 [2]   Go Down
Author Topic: Fast data logger for multiple analog pins  (Read 7035 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 1
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello fat16lib,

Many thanks for the file. Seems to work really well. I was trying to make this fast analog reading with the analoglogger from the sdfat library, which was having quite trouble to make it below 20ms.

Let me make a suggestion for improvement, a code that reads the already present csv files on the SDcard and doesn't overwrite them!

Keep up the good work,
Logged

0
Offline Offline
Edison Member
*
Karma: 64
Posts: 1638
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Movykappa,

Thanks for the suggested improvement.  Many people have suggested other features and "improvements".

The purpose of this examples is to provide a simple base for fast data logging that can easily be modified so I have not complicated the example with the many suggested additions.

I will likely add a function to SdFat to create a new file with rotation numbers.

The function would be used like this:
Code:
  if (!file.createUnique("LOG.CSV")) {
    Serial.println("createUnique failed");
    while (1);
  }

This would create and open a new empty file with name "LOGnn.CSV" where  nn is 00 - 99.  This would prevent writing over existing files.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 40
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello everybody.
 
I decided to create another analog datalogger for my RC aircraft but I need some help and tips for design.
Since I started learning FreeRTOS for Arduino, I realized soon, that the best way is to use an real time application for savings data on a SD card.
 
So here the general specifies:
 
  • Task 1 (“T1”, high priority) reads the 6 analog pins every 4 millisecs and save them in a struct. Then move the struct into a queue;
  • Task 2 (“T2”, lower priority) reads the struct from the queue and save it into a SD Card. This Task runs everytime T1 is blocked;
The biggest problem is that I don t know how to efficiently copy data from the queue to the SD card. Since it could last milliseconds, I thought 2 strategies:
  • to append during T2 the whole queue into the file and then close it => super Problem: it lasts more than 4 - 0.6 = 3.4 milliseconds;
  • to append during T2 just few values from the queue to a static buffer (which can store about 2000-3000 values) and only when the buffer is completely full then transfer all the buffer to the file and close it (during this operation I would stop T1, since I don’t need more values from  the sensors).
 
 
I have no idea, how it is better to implement it. Since I don t have too much time between T1 und T2 I m afraid I can mess all data in the queue. Do you have a much better idea !??!?! Please…you are welcome!!!
 
PS: I know that there are more efficient RTOS out there, like NilRTOS for istance, but please, I m learning now FreeRTOS and I bought the application guide for microcontrollers, so I don t want to get crazy learning another RTOS.


* Bildschirmfoto1.png (15.38 KB, 787x364 - viewed 29 times.)
« Last Edit: June 26, 2013, 12:29:03 pm by wilhem » Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 40
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hello

Ok. I've read the documentation (the examples) of NilRTOS and I understood the idea, to create a buffer, in wich data are going to be stored and read according to the FIFO rule. And both tasks are going to be synchronized to put and read data from the buffer.
maybe it is a better idea, dont know... may I ask, if my idea to put data in  the queue with FreeRTOS functions synchronizing the tasks is better than to create a single buffer like the example in NilRTOS!?!?!??

I would like to understand more....

Thank you very much

Regards
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 1
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

 Hello,

 I tryd to use this sketch for logging with a manual on/of switch on a digital pin instead of serial character.
I tryed simply to replace " while (!Serial.available()) {  "  with    " while (!Pinbutton high()) {   "
but couldn,t get it to work.
any suggestion for this    Where to place the digitalpin read ?

klass

Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2073
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
maybe it is a better idea, dont know... may I ask, if my idea to put data in  the queue with FreeRTOS functions synchronizing the tasks is better than to create a single buffer like the example in NilRTOS!?!?!??

I do not know how it is with Freertos, but I did it with Nilrtos - the NilFIFO.h is a ready class, and it works. I sampled and wrote struct=88bytes of data 333x per second to the sdcard without lost bytes (with 1284p and 14kB ram used).

The FIFO is needed because of the sdcard's write latencies (up to 250ms, typically 10-80ms). So you want write your struct into FIFO with the first task at sampling intervals, with the second one you write the struct from FIFO to the sdcard (when the FIFO not empty). During sdcard's outages the FIFO fills itself, when the sdcard is ready to accept the writes the FIFO empties..

Mind the FIFO ram size depends on struct size and the sampling period and expected latencies. With ie. struct=100bytes and with 200x per second sampling rate and 250ms worst case sdcard's latency the FIFO needs 100x200x0.25=5kBytes of ram.
« Last Edit: October 09, 2013, 04:00:37 pm by pito » Logged

Rapa Nui
Offline Offline
Edison Member
*
Karma: 60
Posts: 2073
Pukao hats cleaning services
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I tryd to use this sketch for logging with a manual on/of switch on a digital pin instead of serial character.
I tryed simply to replace " while (!Serial.available()) {  "  with    " while (!Pinbutton high()) {   "
but couldn,t get it to work.
any suggestion for this    Where to place the digitalpin read ?

Try:
Code:
while( !(digitalRead(buttonPin) == HIGH))
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 1
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi!
I was wondering, would this program be able to log 100,000 samples? I have an Arduino Uno code that is able to log 100,000 samples with a high quality SD card but it can only log  one analog input. I upgraded to a Mega and I was wondering if this multiple analog input logger can go up to 100,000 samples. I only need to log maximum 3 analog inputs.
Thanks!
Logged

0
Offline Offline
Edison Member
*
Karma: 64
Posts: 1638
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

No.  

The AVR ADC can log one pin at 100,000 samples per second but accuracy is reduced to under 8-bits.  It is not possible to log faster.

It is possible to log much faster with modern ARM devices using DMA.  I have been experimenting  with STM32 chips with a 2.4 MSPS ADC.  This ADC can automatically log multiple channels using DMA at (2,400,000/<number-of-channels>). per second.

Edit: Are you sure you are not missing points at 100,000 samples per second?  How are you doing the 100,000 samples per second?
« Last Edit: July 07, 2014, 04:37:15 pm by fat16lib » Logged

Pages: 1 [2]   Go Up
Jump to: