Show Posts
Pages: 1 [2] 3 4 ... 111
16  Using Arduino / Storage / Re: Difference between SD Card and Micro SD Card? on: September 11, 2014, 09:54:23 am
As SuferTim posted while I wrote this:

The resistors on the LC Studio card are not level shifters, just pull to 3.3V.

The full 5V Arduino signals are applied to MOSI, SCK, and SDCS.  This works on some cards because the input protection on the card causes a lot of current to be drawn and the signals never reach 5V.  You may be exceeding the max pin current of 40 ma for arduino pins.

You may damage cards and often the cards will be hot since you are stressing them beyond spec.

So the difference is a random draw of microSD cards that will tolerate the poor design of the LC Studio card and standard cards that fail with the LC Studio card.
17  Using Arduino / Storage / Re: MicroSD Weirdness on: September 06, 2014, 08:58:50 pm
In this older post, you recommend not using flush(), so I'm even more confused now...

I only suggested calling flush instead of close/open which you proposed.  People sometimes call flush after each data record if they fear a crash or power failure.  These flush calls are slow but not nearly as slow as a close followed by an open.

The fastest way to write a file is to open the file once in setup, write data to the file with any number of print and write statements, and then close the file when you are done logging data.

What if I'm willing to bet that the program won't crash?  Do I still have to put in a dataFile.close() in there to save data, or does the program automatically do that?

You must call close since there is no way to know when to automatically close a file on Arduino.  

Once again, all data in the file will be lost if you do not close the file.  The close call updates the directory entry for the file to indicate which clusters contain data and the total byte count in the file.

If you do not close the file the file data will be in "lost clusters".  A scan of the file system on a PC will recover these clusters but will not repair the file which was not closed.
18  Using Arduino / Storage / Re: MicroSD Weirdness on: September 06, 2014, 01:20:13 pm
  I've also read that the SD library has an automatic flush when it reaches 512 bytes, but it shouldn't be relied on because it may fail occasionally, resulting in lost data.

No the "automatic write" does does not fail occasionally.   You will not lose data if you close the file when you are done.  

Do not close and reopen the file.  A call to open can be very costly since a linear search of the entire directory may be required.  A call to flush() is the same as a close followed by an open but minimizes SD accesses.  You only need to call flush() if you fear your program will crash or lose power before you close the file. Data written after the flush call will be lost if the file is not closed.

Close and flush do more than writing the block cache, these calls also update the directory entry which is necessary since the file size is stored in the directory entry. 

Counting 512 byte intervals is of no value in saving data, all data written before a flush/close call is saved.  If you don't close a file, all data written after the last open or flush is lost.
19  Using Arduino / Storage / Re: MicroSD Weirdness on: September 05, 2014, 05:24:24 pm
Opening and closing the file will take a long time and use a lot of power.  Also SD.h may not allow the SD to go into low power sleep mode.  SD.h is based on a very old version of SdFat and has some problems for low power use.

High power use only occurs while a 512 byte block is written to the SD.  If you open a file once and only call flush() when you want to insure data is written to the SD, you will save power.

There are a number of posts in this forum that measure SD power use.

20  Using Arduino / Storage / Re: SdFat update on GitHub on: September 05, 2014, 03:30:02 pm
I posted a beta of SdFat with a faster printField() for the SdFile and SdBaseFile classes

I also added a member function for float:
  int printField (float value, char term, uint8_t prec = 2);

printField() is about four times faster than println(). Here are results from the PrintBenchmark example comparing printField() with println():
Test of println(uint16_t)
Time 6.73 sec
File size 128.89 KB
Write 19.15 KB/sec
Maximum latency: 5168 usec, Minimum Latency: 172 usec, Avg Latency: 328 usec

Test of printField(uint16_t, char)
Time 1.59 sec
File size 128.89 KB
Write 80.91 KB/sec
Maximum latency: 72400 usec, Minimum Latency: 44 usec, Avg Latency: 72 usec

Test of println(uint32_t)
Time 9.91 sec
File size 200.00 KB
Write 20.17 KB/sec
Maximum latency: 5532 usec, Minimum Latency: 464 usec, Avg Latency: 488 usec

Test of printField(uint32_t, char)
Time 2.07 sec
File size 200.00 KB
Write 96.67 KB/sec
Maximum latency: 73676 usec, Minimum Latency: 72 usec, Avg Latency: 95 usec

Test of println(float)
Time 12.88 sec
File size 149.00 KB
Write 11.57 KB/sec
Maximum latency: 5652 usec, Minimum Latency: 548 usec, Avg Latency: 636 usec

Test of printField(float, char)
Time 3.41 sec
File size 149.00 KB
Write 43.64 KB/sec
Maximum latency: 5272 usec, Minimum Latency: 136 usec, Avg Latency: 163 usec


This beta also has software SPI for Teensy 3 and Due.  See the SoftwareSPI.txt file.

SPI transactions have been added. See SPI_Transactions.txt.
21  Using Arduino / Storage / Re: Writing data to csv-file on SD-Card fails on: September 03, 2014, 06:42:11 pm
The old SdFat in SD.h may still have SPI sharing problems.  It should be OK in your tests where you do not access the EEPROM.

To verify you have no SPI problems do the following.

Clone or download SdFat and install it.

Edit SdFatConfig.h and change this symbol from 0 to 2.
 * To enable SD card CRC checking set USE_SD_CRC nonzero.
 * Set USE_SD_CRC to 1 to use a smaller slower CRC-CCITT function.
 * Set USE_SD_CRC to 2 to used a larger faster table driven CRC-CCITT function.
#define USE_SD_CRC 0

Run the bench example

This will verify SPI data transfers using the standard 16-bit SD CRC.

You can also format SD cards using the SdFormatter example

Linux, OS X, and Windows do not conform to the SD standard.   SdFormatter uses proper cluster sizes and alignment so file system boundaries match flash chip programming and erase boundaries.
22  Using Arduino / Storage / Re: Writing data to csv-file on SD-Card fails on: September 02, 2014, 04:27:48 pm
Is your EEPROM or any other device on the SPI bus?  If you have any other SPI device put the chip select pin for the device in output mode and set the pin high.

Check your SD card for file system errors or reformat the SD using the SD Association's formatter

I chose the SD-Library provided by the Arduino-IDE, since I reckon this library should work most reliably.
The standard library should work for you but it is just a wrapper for a very old version of SdFat I wrote five years ago and has a number of missing bug fixes.

One feature in newer versions of SdFat is a CRC check to rule out SPI bus errors.  The Arduino has no hardware CRC so marginal SPI signals can cause this type problem.  You can enable software CRC in newer versions of SdFat by editing SdFatConfig.h.

How fast do you need to log data?  The new SdFat has an example that can log data records at 500 Hz with a time jitter of a few microseconds.  A number of users log accelerometer plus other data using this example.

I also wrote an RamDisk file system that some people use with about a MB of FRAM or SRAM

Sound like your EEPROMS already work.
23  Using Arduino / Storage / Re: MicroSD Weirdness on: September 01, 2014, 03:34:06 pm
SD cards are block devices so it is necessary to read and write entire 512 byte blocks.  Only very limited write access to an SD is possible with out a 512 byte block buffer.

You are close to having memory problems before adding an SD.  You will need a board with more RAM unless you can eliminate some arrays.
24  Using Arduino / Storage / Re: MicroSD Weirdness on: September 01, 2014, 10:27:47 am
You have used most of the 328's RAM on arrays like this:
int irData[64]; //Contains the raw IR data from the sensor
float temperatures[64]; //Contains the calculated temperatures of each pixel in the array
byte eepromData[256]; //Contains the full EEPROM reading from the MLX (Slave 0x50)
int a_ij[64], b_ij[64];
float alpha_ij[64] = {

The SD library requires a 512 byte block buffer plus about 100 additional bytes for file system structures so your program will fail due to lack of RAM.
25  Using Arduino / Storage / Re: SdFat update on GitHub on: August 30, 2014, 09:20:47 pm
I modified SdBaseFile::printField to be correct.  I intend to update SdBaseFile::printField to use routines in FmtNumber.cpp now that I believe the bug in the fast format routines are fixed.

The fastest print routines are now in the new StdioStream class.  this class uses very fast buffering and FmtNumber.cpp.  It will be faster than SdBaseFile::printField even after I modify SdBaseFile::printField  to use FmtNumber.cpp.

Look at and run this example
26  Using Arduino / Storage / Re: Data file larger than 4.3 GB on: August 28, 2014, 08:37:11 pm
SD.h is a wrapper for SdFat and both support FAT32 file systems.

The file size field in a FAT32 directory entries is 32-bits so the max file size is 0XFFFFFFFF bytes or 4,294,967,295 decimal.  There is no way around this limit.

I have looked at Linux file systems like ext3 and the Microsoft exFAT file systems.  I don't think they could run on Uno since these file system are far more complex than FAT32.
27  Using Arduino / Storage / Re: how to read sd file line by line on: August 26, 2014, 08:12:54 pm
Looks like you are using SD.h.  SD.h is a wrapper for an ancient version of SdFat and lacks many of the features in modern versions of SdFat so you can't use fgets().
28  Using Arduino / Storage / Re: how to read sd file line by line on: August 25, 2014, 10:51:39 am
fgets() in SdFat reads a file by line.

Here is a demo program:
// Demo of fgets function to read lines from a file.
#include <SdFat.h>

// SD chip select pin
const uint8_t chipSelect = SS;

SdFat sd;

SdFile file;

// Maximum line length plus space for zero byte.
const size_t LINE_DIM = 50;

char line[LINE_DIM];
// store error strings in flash memory
#define error(s) sd.errorHalt_P(PSTR(s))
void setup(void) {
  size_t n;
  while (!Serial) {}  // Wait for Leonardo

  Serial.println(F("Type any character to start"));
  while ( <= 0) {}
  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
  if (!"TEST.TXT", O_READ)) error("open failed");
  int ln = 1;
  while ((n = file.fgets(line, sizeof(line))) > 0) {
      // Print line number.
      Serial.print(": ");
      if (line[n - 1] != '\n') {
        // Line is too long or last line is missing nl.
        Serial.println(F(" <-- missing nl"));
void loop(void) {}
Here is the test file with no nl character after the last line.
Lines to test fgets.

These lines must be
shorter than the buffer size.
The last line might be missing a new line.
Missing new line char

Here is the output:
Type any character to start
1: Lines to test fgets.
3: These lines must be
4: shorter than the buffer size.
5: The last line might be missing a new line.
6: Missing new line char <-- missing nl

29  Using Arduino / Storage / Re: How to get name of directory when directory name is unknown? SdFat on: August 23, 2014, 09:00:43 am
Here is a function that lists all directories on an SD.  You need to add what you need to find files in each directory.
#include <SdFat.h>
// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
void printDirs(SdBaseFile* dir, uint8_t indent) {
  SdBaseFile subDir;
  // Indent for subdir level
  for (uint8_t i = 0; i < indent; i++) Serial.write(' ');

  // You could use bool getFilename(char* name) here to get the text.
  while (subDir.openNext(dir, O_READ)) {
    if (subDir.isDir()) printDirs(&subDir, indent + 3);
void setup() {
  while (!Serial) {} // wait for Leonardo
  if (!sd.begin(chipSelect)) sd.initErrorHalt();
  printDirs(sd.vwd(), 0);
void loop() {}

Here is a tree list of an SD done on a PC (quote is not spacing these listings correctly):
│   ├───DIR1A
│   └───DIR1B

Here is output from the above program

30  Using Arduino / Storage / Re: Try this super fast analog pin logger on: August 20, 2014, 08:04:11 am
Edit: I suspect using VirtualWire will cause this logger to fail, even if you modify it to use a different timer.  VirtualWire takes too long in its interrupt routine.

Timer1 is used to start the ADC conversion.  Only timer0 or timer1 can be used to trigger the ADC but timer0 is used for mills() and micros().

Timer2 is free but a mod to the VirtualWire library would be required.  This is probably not too difficult since VirtualWire runs on ATtiny85 using timer0 which is 8-bits.  I believe timer2 has all the required features.

If you can live with logging at 500 Hz or less, the new SdFat has a logger that does not use timer1, the LowLatencyLogger example.

The new library is here

Using VirtialWire with any fast logger may cause problems.
Pages: 1 [2] 3 4 ... 111