Pages: [1]   Go Down
Author Topic: Where are my datas?  (Read 674 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 19
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
When you use file.write(), it doesn't write to the card until you flush() or close(). Whenever you open a file, be sure to close it to save your data.

I guess they are in Arduino RAM until i flush or close? So i'm limited with keeping data open (i have from time to time close/flush if i don't want to loss my datas?)? Or am i completly wrong?
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 549
Posts: 46154
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I guess they are in Arduino RAM until i flush or close?
Yes.

Quote
So i'm limited with keeping data open (i have from time to time close/flush if i don't want to loss my datas?)?
The only way to ensure that data is committed to the file is to close it. The tradeoff is that closing, reopening, and relocating the insertion point is slow. Doing so will reduce the rate at which you can write to the file.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 19
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The only way to ensure that data is committed to the file is to close it. The tradeoff is that closing, reopening, and relocating the insertion point is slow. Doing so will reduce the rate at which you can write to the file.

hmm :-/
So it would be best to collect data until RAM is nearly full, then close the file, and open a new file instead of reopening and relocating...
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 549
Posts: 46154
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
So it would be best to collect data until RAM is nearly full, then close the file, and open a new file instead of reopening and relocating...
Depends on whether speed or no-loss-of-data is more important.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 19
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
So it would be best to collect data until RAM is nearly full, then close the file, and open a new file instead of reopening and relocating...
Depends on whether speed or no-loss-of-data is more important.
Speed....AND no-loss-of-data too  smiley-cool
Logged

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1477
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Close() followed by open() is the wrong way to go.  It take a long time to open a file and position to the end.

With SD.h flush() does exactly the same as a close() followed by open().  With SdFat use sync().

SD.h is a wrapper for SdFat so it calls the same close() function as SdFat.

Here is close():

Code:
uint8_t SdFile::close(void) {
  if (!sync())return false;
  type_ = FAT_FILE_TYPE_CLOSED;
  return true;
}

As you can see, close() calls sync() and marks the file closed by setting type_.

You can call flush() or sync() as often as you like but it will cost 2048 bytes of I/O to the SD.  A 512 byte block will be written to update the data on the SD, a 512 byte block containing the directory entry  will be read, updated and written back, then the 512 byte data block will be read back into the block buffer.

I am the author of SdFat which is the base library for the official SD.h library.


Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 19
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Close() followed by open() is the wrong way to go.  It take a long time to open a file and position to the end.
I don't want to open the same file i was closing. close firstfile.txt open secondfile.txt


You can call flush() or sync() as often as you like but it will cost 2048 bytes of I/O to the SD.  A 512 byte block will be written to update the data on the SD, a 512 byte block containing the directory entry  will be read, updated and written back, then the 512 byte data block will be read back into the block buffer.

I'm recording a measuring value every 0.025 sec ~6 hours long in csv format. 
So this would be ~864000 lines.
What would you advise me? Periodically sync?

I am the author of SdFat which is the base library for the official SD.h library.
I know   smiley-wink
Logged

0
Offline Offline
Edison Member
*
Karma: 44
Posts: 1477
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You will need special techniques to record a measurement every 0.025 sec.  SD cards occasionally have very long write latency.  The standard allows up to 200 ms and 100 ms is common.

You can check your card by running the SdFat bench example to find the maximum write latency.

Here is the result for an old blue SanDisk 2GB card with max latency of 0.139 sec:
Quote
Starting write test.  Please wait up to a minute
Write 193.80 KB/sec
Maximum latency: 139360 usec, Avg Latency: 511 usec

Here is the result for a SanDisk 4GB Extreme HD video card with max latency of 0.047 sec.
Quote
Starting write test.  Please wait up to a minute
Write 160.56 KB/sec
Maximum latency: 47028 usec, Avg Latency: 618 usec

The Extreme HD card is about as good as you can do and it is too slow for your measurements in a simple sketch.

I have posted several examples of fast data loggers here http://code.google.com/p/beta-lib/downloads/list.

The file fastLoggerBeta20110802.zip examples uses interrupts for measurements and have extra buffering.

The file ChibiOS20120221.zip uses a real-time OS to run separate threads for measurement and writing to the SD.  These examples also use a buffer queue between threads.
Logged

Pages: [1]   Go Up
Jump to: