Show Posts
Pages: 1 2 [3] 4 5 ... 112
31  Using Arduino / Storage / Re: MicroSD Weirdness on: September 06, 2014, 01:20:13 pm
Quote
  I've also read that the SD library has an automatic flush when it reaches 512 bytes, but it shouldn't be relied on because it may fail occasionally, resulting in lost data.

No the "automatic write" does does not fail occasionally.   You will not lose data if you close the file when you are done.  

Do not close and reopen the file.  A call to open can be very costly since a linear search of the entire directory may be required.  A call to flush() is the same as a close followed by an open but minimizes SD accesses.  You only need to call flush() if you fear your program will crash or lose power before you close the file. Data written after the flush call will be lost if the file is not closed.

Close and flush do more than writing the block cache, these calls also update the directory entry which is necessary since the file size is stored in the directory entry. 

Counting 512 byte intervals is of no value in saving data, all data written before a flush/close call is saved.  If you don't close a file, all data written after the last open or flush is lost.
32  Using Arduino / Storage / Re: MicroSD Weirdness on: September 05, 2014, 05:24:24 pm
Opening and closing the file will take a long time and use a lot of power.  Also SD.h may not allow the SD to go into low power sleep mode.  SD.h is based on a very old version of SdFat and has some problems for low power use.

High power use only occurs while a 512 byte block is written to the SD.  If you open a file once and only call flush() when you want to insure data is written to the SD, you will save power.

There are a number of posts in this forum that measure SD power use.

33  Using Arduino / Storage / Re: SdFat update on GitHub on: September 05, 2014, 03:30:02 pm
I posted a beta of SdFat with a faster printField() for the SdFile and SdBaseFile classes https://github.com/greiman/SdFat-beta.

I also added a member function for float:
Code:
  int printField (float value, char term, uint8_t prec = 2);

printField() is about four times faster than println(). Here are results from the PrintBenchmark example comparing printField() with println():
Quote
Test of println(uint16_t)
Time 6.73 sec
File size 128.89 KB
Write 19.15 KB/sec
Maximum latency: 5168 usec, Minimum Latency: 172 usec, Avg Latency: 328 usec

Test of printField(uint16_t, char)
Time 1.59 sec
File size 128.89 KB
Write 80.91 KB/sec
Maximum latency: 72400 usec, Minimum Latency: 44 usec, Avg Latency: 72 usec

Test of println(uint32_t)
Time 9.91 sec
File size 200.00 KB
Write 20.17 KB/sec
Maximum latency: 5532 usec, Minimum Latency: 464 usec, Avg Latency: 488 usec

Test of printField(uint32_t, char)
Time 2.07 sec
File size 200.00 KB
Write 96.67 KB/sec
Maximum latency: 73676 usec, Minimum Latency: 72 usec, Avg Latency: 95 usec

Test of println(float)
Time 12.88 sec
File size 149.00 KB
Write 11.57 KB/sec
Maximum latency: 5652 usec, Minimum Latency: 548 usec, Avg Latency: 636 usec

Test of printField(float, char)
Time 3.41 sec
File size 149.00 KB
Write 43.64 KB/sec
Maximum latency: 5272 usec, Minimum Latency: 136 usec, Avg Latency: 163 usec

Done!

This beta also has software SPI for Teensy 3 and Due.  See the SoftwareSPI.txt file.

SPI transactions have been added. See SPI_Transactions.txt.
34  Using Arduino / Storage / Re: Writing data to csv-file on SD-Card fails on: September 03, 2014, 06:42:11 pm
The old SdFat in SD.h may still have SPI sharing problems.  It should be OK in your tests where you do not access the EEPROM.

To verify you have no SPI problems do the following.

Clone or download SdFat https://github.com/greiman/SdFat and install it.

Edit SdFatConfig.h and change this symbol from 0 to 2.
Code:
/**
 * To enable SD card CRC checking set USE_SD_CRC nonzero.
 *
 * Set USE_SD_CRC to 1 to use a smaller slower CRC-CCITT function.
 *
 * Set USE_SD_CRC to 2 to used a larger faster table driven CRC-CCITT function.
 */
#define USE_SD_CRC 0

Run the bench example https://github.com/greiman/SdFat/tree/master/SdFat/examples/bench.

This will verify SPI data transfers using the standard 16-bit SD CRC.

You can also format SD cards using the SdFormatter example https://github.com/greiman/SdFat/blob/master/SdFat/examples/SdFormatter/SdFormatter.ino.

Linux, OS X, and Windows do not conform to the SD standard.   SdFormatter uses proper cluster sizes and alignment so file system boundaries match flash chip programming and erase boundaries.
35  Using Arduino / Storage / Re: Writing data to csv-file on SD-Card fails on: September 02, 2014, 04:27:48 pm
Is your EEPROM or any other device on the SPI bus?  If you have any other SPI device put the chip select pin for the device in output mode and set the pin high.

Check your SD card for file system errors or reformat the SD using the SD Association's formatter https://www.sdcard.org/downloads/formatter_4/.

Quote
I chose the SD-Library provided by the Arduino-IDE, since I reckon this library should work most reliably.
The standard library should work for you but it is just a wrapper for a very old version of SdFat I wrote five years ago and has a number of missing bug fixes.

One feature in newer versions of SdFat is a CRC check to rule out SPI bus errors.  The Arduino has no hardware CRC so marginal SPI signals can cause this type problem.  You can enable software CRC in newer versions of SdFat by editing SdFatConfig.h.

How fast do you need to log data?  The new SdFat has an example that can log data records at 500 Hz with a time jitter of a few microseconds.  A number of users log accelerometer plus other data using this example.

I also wrote an RamDisk file system that some people use with about a MB of FRAM or SRAM https://github.com/greiman/RamDisk.

Sound like your EEPROMS already work.
36  Using Arduino / Storage / Re: MicroSD Weirdness on: September 01, 2014, 03:34:06 pm
SD cards are block devices so it is necessary to read and write entire 512 byte blocks.  Only very limited write access to an SD is possible with out a 512 byte block buffer.

You are close to having memory problems before adding an SD.  You will need a board with more RAM unless you can eliminate some arrays.
37  Using Arduino / Storage / Re: MicroSD Weirdness on: September 01, 2014, 10:27:47 am
You have used most of the 328's RAM on arrays like this:
Code:
int irData[64]; //Contains the raw IR data from the sensor
float temperatures[64]; //Contains the calculated temperatures of each pixel in the array
byte eepromData[256]; //Contains the full EEPROM reading from the MLX (Slave 0x50)
int a_ij[64], b_ij[64];
float alpha_ij[64] = {

The SD library requires a 512 byte block buffer plus about 100 additional bytes for file system structures so your program will fail due to lack of RAM.
38  Using Arduino / Storage / Re: SdFat update on GitHub on: August 30, 2014, 09:20:47 pm
I modified SdBaseFile::printField to be correct.  I intend to update SdBaseFile::printField to use routines in FmtNumber.cpp now that I believe the bug in the fast format routines are fixed.

The fastest print routines are now in the new StdioStream class.  this class uses very fast buffering and FmtNumber.cpp.  It will be faster than SdBaseFile::printField even after I modify SdBaseFile::printField  to use FmtNumber.cpp.

Look at and run this example https://github.com/greiman/SdFat/blob/master/SdFat/examples/StdioBench/StdioBench.ino.
39  Using Arduino / Storage / Re: Data file larger than 4.3 GB on: August 28, 2014, 08:37:11 pm
SD.h is a wrapper for SdFat and both support FAT32 file systems.

The file size field in a FAT32 directory entries is 32-bits so the max file size is 0XFFFFFFFF bytes or 4,294,967,295 decimal.  There is no way around this limit.

I have looked at Linux file systems like ext3 and the Microsoft exFAT file systems.  I don't think they could run on Uno since these file system are far more complex than FAT32.
40  Using Arduino / Storage / Re: how to read sd file line by line on: August 26, 2014, 08:12:54 pm
Looks like you are using SD.h.  SD.h is a wrapper for an ancient version of SdFat and lacks many of the features in modern versions of SdFat so you can't use fgets().
41  Using Arduino / Storage / Re: how to read sd file line by line on: August 25, 2014, 10:51:39 am
fgets() in SdFat reads a file by line.

Here is a demo program:
Code:
// Demo of fgets function to read lines from a file.
#include <SdFat.h>

// SD chip select pin
const uint8_t chipSelect = SS;

SdFat sd;

SdFile file;

// Maximum line length plus space for zero byte.
const size_t LINE_DIM = 50;

char line[LINE_DIM];
//------------------------------------------------------------------------------
// store error strings in flash memory
#define error(s) sd.errorHalt_P(PSTR(s))
//------------------------------------------------------------------------------
void setup(void) {
  size_t n;
  Serial.begin(9600);
  while (!Serial) {}  // Wait for Leonardo

  Serial.println(F("Type any character to start"));
  while (Serial.read() <= 0) {}
 
  if (!sd.begin(chipSelect, SPI_HALF_SPEED)) sd.initErrorHalt();
 
  if (!file.open("TEST.TXT", O_READ)) error("open failed");
 
  int ln = 1;
  while ((n = file.fgets(line, sizeof(line))) > 0) {
      // Print line number.
      Serial.print(ln++);
      Serial.print(": ");
      Serial.print(line);
      if (line[n - 1] != '\n') {
        // Line is too long or last line is missing nl.
        Serial.println(F(" <-- missing nl"));
      }
  }
  Serial.println(F("\nDone"));
}
void loop(void) {}
Here is the test file with no nl character after the last line.
Quote
Lines to test fgets.

These lines must be
shorter than the buffer size.
The last line might be missing a new line.
Missing new line char

Here is the output:
Quote
Type any character to start
1: Lines to test fgets.
2:
3: These lines must be
4: shorter than the buffer size.
5: The last line might be missing a new line.
6: Missing new line char <-- missing nl

Done
42  Using Arduino / Storage / Re: How to get name of directory when directory name is unknown? SdFat on: August 23, 2014, 09:00:43 am
Here is a function that lists all directories on an SD.  You need to add what you need to find files in each directory.
Code:
#include <SdFat.h>
// SD chip select pin
const uint8_t chipSelect = SS;

// file system object
SdFat sd;
//------------------------------------------------------------------------------
void printDirs(SdBaseFile* dir, uint8_t indent) {
  SdBaseFile subDir;
  // Indent for subdir level
  for (uint8_t i = 0; i < indent; i++) Serial.write(' ');

  // You could use bool getFilename(char* name) here to get the text.
  dir->printName(&Serial);
  Serial.println();
  dir->rewind();
  while (subDir.openNext(dir, O_READ)) {
    if (subDir.isDir()) printDirs(&subDir, indent + 3);
    subDir.close();
  }
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  while (!Serial) {} // wait for Leonardo
  delay(1000);
  
  if (!sd.begin(chipSelect)) sd.initErrorHalt();
  printDirs(sd.vwd(), 0);
  Serial.println("Done!");
}
void loop() {}

Here is a tree list of an SD done on a PC (quote is not spacing these listings correctly):
Quote
H:.
├───DIR1
│   ├───DIR1A
│   └───DIR1B
└───DIR2
    ├───DIR2A
    └───DIR2B
        ├───DIR2B1
        └───DIR2B2

Here is output from the above program
Quote
/
   DIR1
      DIR1A
      DIR1B
   DIR2
      DIR2A
      DIR2B
         DIR2B1
         DIR2B2
Done!

43  Using Arduino / Storage / Re: Try this super fast analog pin logger on: August 20, 2014, 08:04:11 am
Edit: I suspect using VirtualWire will cause this logger to fail, even if you modify it to use a different timer.  VirtualWire takes too long in its interrupt routine.

Timer1 is used to start the ADC conversion.  Only timer0 or timer1 can be used to trigger the ADC but timer0 is used for mills() and micros().

Timer2 is free but a mod to the VirtualWire library would be required.  This is probably not too difficult since VirtualWire runs on ATtiny85 using timer0 which is 8-bits.  I believe timer2 has all the required features.

If you can live with logging at 500 Hz or less, the new SdFat has a logger that does not use timer1, the LowLatencyLogger example.

The new library is here https://github.com/greiman/SdFat.

Using VirtialWire with any fast logger may cause problems.
44  Using Arduino / Storage / Re: Program Stuck at SD.begin(4) on: August 19, 2014, 09:06:46 am
Do you have anything in loop() ?   loop() will execute even if the SD fails.

Try replacing these  return statements in startup() with  a forever loop.  And add a Serial.flush.  This won't solve the problem but may allow the entire message to print.
Code:
if (!SD.begin(4)) {
        Serial.println("ERROR - SD card initialization failed!");
       for (;;);    // init failed  <-----------------------------------forever loop
    }
    Serial.println("SUCCESS - SD card initialized.");
    if (!SD.exists("index.htm")) {
        Serial.println("ERROR - Can't find index.htm file!");
        for (;;);  // can't find index file <---------------------------forever loop
    }
    Serial.println("SUCCESS - Found index.htm file.");
    Serial.flush();  // <-------------------------------------------flush serial print data

45  Using Arduino / Storage / Re: Using SdFat with String on: August 18, 2014, 12:03:31 pm
The following will get the file name for myFile and place it in an Arduino String.
Code:
  SdFile myFile;
  char tmp[13];

  // code to open mfFile goes here

  myFile.getFilename(tmp);
  String name = tmp;

  // verify that the file name is in the String.
  Serial.println(name);

The output for a test case is:
Quote
TEST.TXT
Pages: 1 2 [3] 4 5 ... 112