Show Posts
Pages: 1 2 [3] 4 5 ... 111
31  Using Arduino / Storage / Re: SD.write taking 3 seconds on: August 16, 2014, 09:01:02 am
Does the file get created by the open?

The first write may need to allocate a cluster.  This can cause a scan of the FAT, File Allocation Table, and can take time to find the first free cluster if your card has lots of data.

If you formatted the card with a small cluster size it can take a very long time.  Be sure to use SD Formatter, not an OS utility.
32  Using Arduino / Storage / Re: Which microSD cards to use for low current & low power consumption? on: August 16, 2014, 08:15:05 am
Looks like pito is correct, old cards use less power.

I did some tests with two cards, an older SanDisk 1 GB SD and a new SanDisk 16 GB high end microSDHC.

Manufacturer ID: 0X3
Product: SE16G
Version: 8.0
Serial number: 0X3752B221
Manufacturing date: 10/2013

cardSize: 15931.54 MB (MB = 1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true

Manufacturer ID: 0X3
Product: SD01G
Version: 8.0
Serial number: 0XFB5C4F70
Manufacturing date: 12/2006

cardSize: 1015.81 MB (MB = 1,000,000 bytes)
flashEraseSize: 32 blocks
eraseSingleBlock: true

I ran the new dataLogger example in the 20140806 version of SdFat

 I used a Teensy 3.1 so there would be no level shifters or regulators.

First I put a 4.7 ohm resistor in the ground line and looked at the voltage on a scope.  See the three attached traces. The scale is 100 mv/div so with the 4.7 ohm resistor, the scale is about 21 ma/div.

There are two views of the new trace,  the full trace at 2 ms/div and a magnified portion at 500 us/div.  Notice that the current stays higher, about 10 ma for about 6 extra ms.

If you look at the two 500 us/div traces, the write current in the trace is higher for the new card.

Edit: I added a third version of the new trace at 2 usec/div for fun.

I also put a handheld DMM in the 3.3V line to check idle current.  I found that the cards went through at least two states.  The new card immediately drops to about 500 uamp then sleeps at about 70 uamp .  The old card immediately drops to about 200 uamp and sleeps at 60 uamp.

The dataLogger example is a current hog since it calls sync() after every data point.  That will cause at least two reads and two writes for each data point.  If you remove the sync calls after each point, the power use will drop a huge amount.
33  Using Arduino / Storage / Re: Which microSD cards to use for low current & low power consumption? on: August 15, 2014, 12:05:51 pm
Any experience with the old Nokia  microSD cards?

I have not tested Nokia cards but in that era camera and phone companies slapped there brand on cards made by some other company.

Here a "Canon card" is really a Panasonic card:
Manufacturer ID: 0X1
Product: S016B
Version: 4.5
Serial number: 0X35C4B589
Manufacturing date: 9/2005

Run the SdInfo example, often it will give you a clue about what you really have.
34  Using Arduino / Storage / Re: SdCard - reverse engineered on: August 15, 2014, 11:27:39 am
Wow, you don't give up easily.

The "spooks" like NSA and CIA have forensic programs that reconstruct file systems almost automatically.

When I was a mint PhD physicist, I did some work on hydrodynamic modeling programs at a "weapons lab".  When I finished, they shredded my computer.  With people like you around, I see why.

My work was to support the modeling of weapons after the Underground Test Ban.  Before the ban, some computers including the keyboard were put underground with the test device.  Bet that would stop you.

35  Using Arduino / Storage / Re: Which microSD cards to use for low current & low power consumption? on: August 15, 2014, 11:11:32 am
You may read the datasheet and then find you card is not even close.  This happened to me recently when my SanDisk card turned out to be counterfeit   Your article about fakes is true, I now keeps a collection of fakes where I have been ripped off.

This was from an Amazon seller.  If you buy an older smaller card now, the chances are very high the card will be fake.  Forget it on ebay it's a coin toss.  I don't think many sellers know they are pushing fake cards.

Major manufactures no longer produce standard, 2 GB or less, cards so the market for these is full of fakes.

Most cards are now used in phones and power consumption is becoming a bigger issue.  I recently bought about a dozen microSDHC cards  I should test this batch for power use.

Once again, the datasheet won't tell you much for use on Arduino.  I recently did tests with cards on SDIO using large multi-block writes on a high end micro.  The performance is totally different than on SPI with the same card.  I get read/write speeds of 20 MB/sec on SDIO so there can't be much extra flash programming.
36  Using Arduino / Storage / Re: Which microSD cards to use for low current & low power consumption? on: August 14, 2014, 06:02:53 pm
You will never find a simple answer.  The amount of power used depends on how much flash is programmed.

The amount of flash programming depends on the flash controller and your write pattern.  Modern cards have very large pages, often some multiple of 16 KB.

When you write a single 512 byte block with a single block write command, the entire flash page must be programmed so the controller must move existing data to a new flash page.  Wear leveling algorithms also cause extra data movement.

I could continue on forever but the short answer is SD cards were not designed for Arduino use.  The amount of flash programming in the card is much greater than the amount you write.

I have tested many cards and there is not a "best" card for all applications.

You need to do testing with your application.

Another key facror in power usage is card idle current.  SD.h has a bug that can prevent cards from sleeping in low power mode.

I wrote the old version of SdFat used in SD.h and it has not been updated since I fixed the bug.
37  Using Arduino / Storage / Re: Teensy 3.1 / Data Logging / SD Card / Latency / Non-Blocking SDFAT Library on: August 11, 2014, 06:23:03 pm
QuickStart has this warning:
// Normally the SdFat class is used in applications in place
// of Sd2Card, SdVolume, and SdFile for root.
// These internal classes are used here to diagnose problems.
Sd2Card card;
SdVolume volume;
SdFile root;

SdInfo uses these because it is also intended to diagnose problems.  It must work with an SD card that is not formatted.

Look at something like the ReadWriteSdFat example.
38  Using Arduino / Storage / Re: Teensy 3.1 / Data Logging / SD Card / Latency / Non-Blocking SDFAT Library on: August 11, 2014, 05:06:05 pm
SdSpiMK20DX128.cpp works for both Teensy 3.0 and Teensy 3.1.  It is Paul's version.  Ignore the file name it dates from when only Teensy 3.0 existed.

I can do card.init, vol.init, those basics. but card.begin() fails,

Why are you doing this?  You should not be using these calls.

You should only need sd.begin() in your code.

Does the bench example work?

The results above are running the GitHub version on a Teensy 3.1 with no mods.

Have you tried any of the SdFat examples ?
39  Using Arduino / Storage / Re: Teensy 3.1 / Data Logging / SD Card / Latency / Non-Blocking SDFAT Library on: August 11, 2014, 01:25:09 pm
Are you using this version of SdFat?

There is a mod in SdSpiMK20DX128.cpp by Paul Stoffregen that fixes SPI problems on Teensy 3.0 and 3.1 with newer versions of the Teensy IDE.  This fix may help.

Another option is to edit SdFatConfig.h and use the standard SPI.h library.  Change this line.
 * Force use of Arduino Standard SPI library if USE_ARDUINO_SPI_LIBRARY
 * is nonzero.

I tested this version on Teensy 3.1 using the bench example with 32 KB reads and writes.  about 2200 KB/sec write and 2500 KB/sec read.

Use a freshly formatted SD for best performance.

Type any character to start
Free RAM: 28315
Type is FAT32
Card size: 15.93 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
Product: SE16G
Version: 8.0
Serial number: 0X3752B221
Manufacturing date: 10/2013

File size 5 MB
Buffer size 32768 bytes
Starting write test, please wait.

write speed and latency

Starting read test, please wait.

read speed and latency


Here is the LowLatencyLogger example logging four analog pins per sample at 2500 samples/sec.

Creating new file
Erasing all data
Logging - type any character to stop
Truncating file
File renamed: DATA03.BIN
Max block write usec: 200
Record time sec: 9.747
Sample count: 24365
Samples/sec: 2499.74
Overruns: 0

40  Using Arduino / Storage / Re: SD.h library on: August 07, 2014, 01:37:04 pm
SD.h is old and buggy.  It is a wrapper for an old version of SdFat that I wrote in 2009.

You added a bug to the example that caused a memory leak so it is now really broken.

      Serial.println(entry.size(), DEC);
     entry.close();  <-------- Don't remove this, it frees memory.

The example stopped listing files when free memory was exhausted.

You don't need these but they won't hurt.
      // no more files
       entry.rewindDirectory();  <--------------- Not needed
       entry.close();  <-------------------------  Not needed, the file didn't open so there is no memory.

This is what was listed on the first call.  Files are not sorted so it is correct.
001.TXT      33
002.TXT      15
003.TXT      15
004.TXT      13
005.TXT      15
006.TXT      15
007.TXT      15
008.TXT      15
009.TXT      185
   006.TXT      15
   007.TXT      15
   008.TXT      15
   009.TXT      185
   001.TXT      33
   002.TXT      15
   003.TXT      15
   004.TXT      13
   005.TXT      15

Here is where you ran out of memory on the second call.
001.TXT      33
002.TXT      15
003.TXT      15
**nomorefiles**  <------------------ Memory gone

Here is the third call, no memory so no list.

SdFat does not use dynamic memory.  This happen due to poor design of the SD.h wrapper which allocates memory when you open a file and frees it when you close a file.

I just posted a new version of SdFat  It has an ls() function so you don't need to write a list files function.  Download it and look at the examples.  It's far more complex than SD.h, that's why the Arduino company wrote the SD.h wrapper to reduce the number of API functions.
41  Using Arduino / Storage / SdFat update on GitHub on: August 05, 2014, 02:21:20 pm
I have posted a new version of SdFat on GitHub

This version of SdFat has a large number of internal modifications.  I have done a number of tests but I suspect it may break some applications.  Please try this SdFat library and report any problems.

Try the LowLatencyLogger example, it uses one of the new features.  It is easily modifiable and does not use an RTOS or tricky ISR.  I was able to log four analog pins on an Uno at 500 Hz and at 5000 Hz on a Due.  I used a low cost class 4 SanDisk card.

Here are the three functions I used.  You need to modify these for your needs.
const uint8_t ADC_DIM = 4;
struct data_t {
  unsigned long time;
  unsigned short adc[ADC_DIM];

// Acquire a data record.
void acquireData(data_t* data) {
  data->time = micros();
  for (int i = 0; i < ADC_DIM; i++) {
    data->adc[i] = analogRead(i);

// Print a data record.
void printData(Print* pr, data_t* data) {
  for (int i = 0; i < ADC_DIM; i++) {

// Print data header.
void printHeader(Print* pr) {
  for (int i = 0; i < ADC_DIM; i++) {

I have added several new examples, deleted a few, and modified many of the rest.

The StdioBench example demonstrates another feature I am working on, stdio style I/O.  The StdioStream class has stdio style buffering with a configurable amount of ungetc() push-back to make parsing easier.

Here are results comparing Arduino Print with StdioStream formatting on Uno.
Starting test
uint8_t 0 to 255, 100 times
fileSize: 116500
print millis: 6180
stdio millis: 837
ratio: 7.38

uint16_t 0 to 20000
fileSize: 128890
print millis: 6500
stdio millis: 863
ratio: 7.53

uint32_t 0 to 20000
fileSize: 128890
print millis: 6475
stdio millis: 962
ratio: 6.73

uint32_t 1000000000 to 1000010000
fileSize: 120000
print millis: 5771
stdio millis: 866
ratio: 6.66

float nnn.ffff, 10000 times
fileSize: 100000
print millis: 9366
stdio millis: 1498
ratio: 6.25
42  Using Arduino / Storage / Re: Suggest a temporary storage? on: August 04, 2014, 08:29:51 am
You will still get over 30 years of use from a FRAM chip.

28 sensors at 200/sec is only 5,600 values per second.  That's not a lot of data for FRAM. You could write 40 bytes per value for 30 years.
43  Using Arduino / Storage / Re: can't initialize SDcard anymore - Ard. UNO on: August 04, 2014, 08:22:04 am
The attached zip file contains a new development version of SdFat with a new version of the QuickStart.ino example.

Please place the SdFat folder in you libraries folder, run the QuickStart example, and post the output.

QuickStart.ino is designed to diagnose initialization problems.  QuickStart is in the SdFat/examples/QuickStart folder.

I wrote the old version of SdFat that is used in SD.h but it is old and buggy so I don't want to diagnose your problem using SD.h.
44  Using Arduino / Storage / Re: Suggest a temporary storage? on: August 04, 2014, 06:32:59 am
Wow!  You must plan on logging a lot of data.  

The Fuji FRAM is good for over a trillion read/write cycles so the chip is good for over 100,000 TB if you don't burn one location.

If you fill it once a second, it will last for  1,000,000,000/(365*24*3600)  = 31 years.

Te 23LC1024 is also a good solution.  I use them also as a RAM disk
45  Using Arduino / Storage / Re: Suggest a temporary storage? on: August 03, 2014, 05:07:20 pm
FRAM might work.  Here is a chip I like

The 256KB chip is 3.3V and would be easy to interface.

I don't know how much power it would draw while writing at 8 MHz.  I expect less than at 25 MHz SPI.  It is low power when idle.  Write time is very short at SPI bus speeds.
Operating power supply current 10.6 mA (Max@25 MHz)
Standby current 35 μA (Typical), 150 μA (Max)
Sleep current 10 μA (Max)
Pages: 1 2 [3] 4 5 ... 111