Show Posts
Pages: [1] 2 3 ... 99
1  Using Arduino / Storage / Re: (several...) SD just, simply, not working! on: Today at 05:31:10 pm
I worry that the problem may not be your micro SD card.  It is very rare for a card to work in 4-bit mode and fail in SPI mode.
2  Using Arduino / Storage / Re: (several...) SD just, simply, not working! on: Today at 04:42:08 pm
I Have good luck with SanDisk and Transcend micro SD cards.  I don't like Kingston but I haven't tried recent cards.

Don't spend a lot on high end cards.  They are optimized for the 4-bit mode of phones. Only SPI mode matters.

I use this 8 GB class 4 card www.amazon.com/gp/product/B000WH6H1M.  Here is a benchmark.
Quote
Type is FAT32
File size 5MB
Buffer size 100 bytes
Starting write test.  Please wait up to a minute
Write 174.54 KB/sec
Maximum latency: 303804 usec, Minimum Latency: 84 usec, Avg Latency: 567 usec

Starting read test.  Please wait up to a minute
Read 310.96 KB/sec
Maximum latency: 2812 usec, Minimum Latency: 84 usec, Avg Latency: 315 usec

Here is a 4GB class 6 Transcend card http://www.amazon.com/Transcend-Class-microSDHC-Memory-TS4GUSDHC6/dp/B0013DXVYK/. This is an old card so you may want to buy a newer class 10 card.
Here are performance numbers
Quote
Type is FAT32
File size 5MB
Buffer size 100 bytes
Starting write test.  Please wait up to a minute
Write 201.74 KB/sec
Maximum latency: 197332 usec, Minimum Latency: 84 usec, Avg Latency: 490 usec

Starting read test.  Please wait up to a minute
Read 308.20 KB/sec
Maximum latency: 2888 usec, Minimum Latency: 84 usec, Avg Latency: 318 usec
I plan on trying this card http://www.amazon.com/Transcend-Class-microSDHC-Memory-TS16GUSDHC10/dp/B004TA0AZM in 8 and 16 GB.


Here is the result for a 4 GB class 4 micro SD Kingston card..
Quote
Type is FAT32
File size 5MB
Buffer size 100 bytes
Starting write test.  Please wait up to a minute
Write 70.97 KB/sec
Maximum latency: 188880 usec, Minimum Latency: 84 usec, Avg Latency: 1403 usec

Starting read test.  Please wait up to a minute
Read 210.81 KB/sec
Maximum latency: 4472 usec, Minimum Latency: 84 usec, Avg Latency: 468 usec
3  Using Arduino / Storage / Re: File System for external EEPROM on: Today at 08:32:32 am
The RamDisk library in this post http://forum.arduino.cc/index.php?topic=229485 can be used with any memory device.

You need to write a class with these two member functions.
Code:
  virtual bool read(uint32_t address, void *buf, size_t nbyte) = 0;
  virtual bool write(uint32_t address, const void *buf, size_t nbyte) = 0;

The library has no cache since it is designed for RAM.  For an EEPROM like the Microchip 25LC1024, it might wise to implement a 256 byte page cache in the EEPROM class.  The filesystem library has a sync() function that could call the the cache flush function at the proper time.

I may see if I have a EEPROM device I can play with.
4  Using Arduino / Storage / Re: Acessing SD card without using Sd.h on: April 22, 2014, 06:16:27 am
Quote
New to programming. Need help accessing SD Card directly from SPI without using Sd.h or any custom made libraries.
I wouldn't recommend that you try this but here is what you need to do.

Read this https://www.sdcard.org/downloads/pls/simplified_specs/part1_410.pdf first.  The SD SPI protocol is covered in section 7.

Then write your own code to access the SD.
5  Using Arduino / Storage / Re: SD Interface Problem Solved on: April 21, 2014, 04:10:14 pm
A Large value pull-up on MISO won't hurt but it is not needed.   I bet you have another problem that is not really fixed.

I recently had a user make a similar claim but the real problem was chip select was floating due to bad wiring.  A pull-up on a data line quieted noise on the nearby floating chip select and it went low long enough for the SD to initialize.  The problem came back when lots of data was read from the SD.  The 50 K internal SD CS pull-up resistor doesn't control chip select on most modules because level shifters over power it.

Quote
The SD card pads are bi-directional with tri-state drivers, and are in a high impedance state until enabled by the SD card, which may not be when the CS line is asserted as the card is not in SPI mode initially. Leaving any digital signal in a floating, unknown, state is not a good idea.

SPI should not have pull-ups or pull-downs on MISO, MOSI, or SCK. There is not a default state since there are four SPI modes and the SD is not the only device on the bus. Why would you pull MISO to 3.3 V on a 5 V system?

Atmel doesn't suggest pull-anything resistors in the processor datasheet. In fact Atmel suggests not using pull-ups/downs in some configurations.

The fact that pull-ups are not required is often stated as an advantage of SPI.
Quote
Typically lower power requirements than I²C or SMBus due to less circuitry (including pull up resistors)

Quote
It probably works without the pull-ups most of the time, but I found a case where it doesn't, and that's why pull-ups are required, so it will work all of the time.
I don't believe you found a case but you may be the first.   I wrote SdFat, which is also the base for the Arduino SD.h library.  You are not the first to make this type of claim.  This type of "fix" means there is another problem that is the real cause and it will likely bite again.

I was involved with Limor Fried on design of Adafruit SD modules and I believe her design is sound and no pull-ups are needed.
 
Read the SD association's SD "Physical Layer Simplified Specification".  The only mention of pull-up for SPI is the 50 K card detect/CS pullup that must be driven low during initialization to select SPI mode.  This is not the DO pin in SPI mode.

There is mention of pullups for the new UHS-II pins 10-17 but these are not relevant here.

6  Using Arduino / Storage / Re: What happens when a SD card full on: April 21, 2014, 02:21:08 pm
Quote
Never seen either used, did you mean Gigabytes?
Most programmers don't have a clue about SI vs IEC Standard Names.

The correct word for 10^9 bytes is gigabyte and for 2^30 bytes it is gibibyte.   I am not very careful when I use GiB or GB.   When I wrote SdFat I did the following to point out storage is decimal.
Quote
cardSize: 3965.19 MB (MB = 1,000,000 bytes)

Quote
fat16 formating type, have a limitation of storage per file which i believe is 4GB,

FAT16 volumes are limited to 65,524 clusters and on SD cards, the max cluster size is 32,768 bytes.  So you can't have a file larger than 2,147,090,432 bytes on a properly formatted FAT16 SD card.  SDHC cards that are larger than 2,147,090,432 bytes must be formatted FAT32.

The FAT file size limit is 4,294,967,295 bytes, one short of 4 GiB.  For disks on Windows 2000 and later, FAT16 volumes  can have 65,536 byte clusters so you can have a 4,294,180,864 byte file on a FAT16 Windows disk.

7  Using Arduino / Storage / Re: What happens when a SD card full on: April 21, 2014, 11:33:50 am
I will try to be serious and avoid using bites per mouth.  So this is about the gibibyte (GiB)  which is 2^30 bytes and the gigabyte (GB) which is 10^9 bytes.  See http://physics.nist.gov/cuu/Units/binary.html.

Quote
Well each saved log character is a byte and 16GB are 17.179.869.184 bytes
SD card capacity is specified as decimal GB just like disk drives.  17.179.869.184 is 16 gibibytes.

A 16 GB card has about 16,000,000,000 bytes.  I checked a 16 GB micros SD and it had 31,291,392 blocks (512 bytes/block) or 16,021,192,704 bytes, not 2^34 bytes.

None of the above matters much for the original question:
Quote
the sketch is just opening a file adding a new line and closing it again. the plan is to do this for a long period of time.

The maximum size of a FAT16/FAT32 file is 4,294,967,295 bytes (4 GiB - 1).  So when the file reaches that size, an error should be returned.

SdFat has a bug and a corrupt file will result.  I used a max size of 4 GiB which overflows the 32-bit file size.

I will fix the next version of SdFat but the old version of SdFat in SD.h will not get fixed.

It's unlikely programmers will ever use the NIST units:
Code:
Examples and comparisons with SI prefixes
one kibibit 1 Kibit = 2^10 bit = 1024 bit
one kilobit 1 kbit = 10^3 bit = 1000 bit
one mebibyte 1 MiB = 2^20 B = 1 048 576 B
one megabyte 1 MB = 10^6 B = 1 000 000 B
one gibibyte 1 GiB = 2^30 B = 1 073 741 824 B
one gigabyte 1 GB = 10^9 B = 1 000 000 000 B
8  Using Arduino / Storage / Re: (several...) SD just, simply, not working! on: April 21, 2014, 07:10:50 am
Quote
Quite strange though, as it's apparently functional for my daily basis needs.

This is can happen.  SD cards function in two entirely different modes. , SPI mode and SDIO mode.  1-bit SPI mode uses a different protocol than 4-bit SDIO mode.  Few people use cards in SPI mode and a few cards just don't work correctly in SPI mode.

You problem is not a formatting problem.  SD error: c,ff  is at the lowest SPI protocol level.
9  Using Arduino / Storage / Re: SD Interface Problem Solved on: April 20, 2014, 11:36:57 am
Quote
I did a bit of research and found SD specs require pull-ups of 10k to 100k.

SD cards don't require pull-ups.  The lines are push-pull driven.  When the card is selected, the card will drive the output line.

A pull-up on the card's data output can be helpful to prevent the line from floating when no card is present.  This prevent noise from confusing card initialization when no card is present.

The Adafruit microSD breakout works fine with out additional pull-ups.  Have you posted this problem and  "solution"  on the Adafruit forum?

10  Using Arduino / Storage / Re: SD logger - with several sensors - not fast enough on: April 20, 2014, 11:28:44 am
Quote
what do you do with the data?
I write it to the SD card.

Quote
do you take a number of samples and average for one reading?
No I write all samples to the SD card.

Quote
how much RAM do you use for storage?
RAM is only used for temporary buffering.

Quote
My example was simply to show that the limiting factor was the serial comms,
You can log data at much higher rates over serial than you are achieving.

Quote
what sort of meta data do you capture?
Metadata is a fairly ambiguous term if you go beyond the definition "Metadata is data about other data".  What do you mean by this statement? 
11  Using Arduino / Storage / Re: SD logger - with several sensors - not fast enough on: April 20, 2014, 06:48:28 am
It is possible to log much faster.

See this http://forum.arduino.cc/index.php?topic=228549.msg1650827#msg1650827.

Here is a table of maximum sample rate vs number of analog pins in a sample.  A Mega was used for more than six pins.
Code:
ADC clock kHz
125 250 500 1000
pins
1 7692 14286 25000 40000
2 3810 6667 11111 16667
3 2572 4790 8421 13559
4 1942 3636 6452 10526
5 1559 2930 5229 8602
6 1303 2454 4396 7273
7 1119 2111 3791 6299
8 980 1852 3333 5556
9 872 1649 2974 4969
10 786 1487 2685 4494
11 715 1354 2446 4103
12 656 1242 2247 3774
13 606 1148 2078 3493
14 563 1067 1932 3252
15 525 996 1806 3042
16 493 935 1695 2857

A RTOS is more general solution.  See https://code.google.com/p/rtoslibs/.  Look at the NilRTOS  nilSdLogger example.

12  Using Arduino / Storage / Re: A tiny printf style library for Serial, SD.h, SdFat. on: April 19, 2014, 01:08:04 pm
Quote
why is it so much faster?

It isn't faster.  I just used micros for something to print so it is not a timing.

I do have a future enhancement to SdFat that will be 5-10 times faster for print.  Much of that will be due to stdio style buffering.  I will also use your fast divide by 10.

About 40 years ago stdio provided a huge speedup in Unix.  The disk drives had a max seek time 0f 55 ms so the read/write system calls were very slow.  I was at Berkeley when BSD Unix was developed on the PDP-11.  I have been looking at back at a release of that system, looks crude now.

Quote
1 call instead of 4 ?
This is why I like the printf style API.
13  Using Arduino / Storage / A tiny printf style library for Serial, SD.h, SdFat. on: April 19, 2014, 11:26:38 am
I have attached a tiny library, MinPrintf, that provides simple printf style functionality for any class derived from the Arduino Print class.  I include examples for Serial, SD.h , and SdFat.

I wrote MinPrintf as a prototype for a very complete version of printf that implements most of the C11/C++2013 standard.  I found this tiny library useful for many apps since it uses very little flash/SRAM  but has key printf functionality.

The printf format string can be stored in flash using the F() macro.

I have not included floating point since this always makes the library very large and is not too useful without precision support.  You can still print floating point with the Arduino print(val, prec) member function.

I will post the large complete printf library later.

Here is a demo example comparing Arduino Print with MinPrintf
Code:
// Demo of MinPrintf
#include <MinPrintf.h>

// Serial MinPrintf object.
MinPrintf mpr(&Serial);
//------------------------------------------------------------------------------
// MinPrintf
void min_print() {
  mpr.printf(F("time: %lu usec, adc: %u\n"), micros(), analogRead(5));
}
//------------------------------------------------------------------------------
// Arduino print.
void ArduinoPrint() {
  Serial.print(F("time: "));
  Serial.print(micros());
  Serial.print(F(" usec, adc: "));
  Serial.println(analogRead(5));
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  min_print();
  ArduinoPrint();
}
void loop() {}

Here is the output:
Quote
time: 60 usec, adc: 1023
time: 884 usec, adc: 1023
14  Using Arduino / Storage / Re: Iterating through a Directory and storing the maximium file number as an integer on: April 16, 2014, 11:48:17 am
Here is a very fast function to find the "maximium file number as an integer"  It was developed for a user with a huge number of files so one pass through the directory was required.
Code:
#include <SD.h>
const uint8_t chipSelect = 10;
//------------------------------------------------------------------------------
// Return -2 if error, -1 not found, else max file number.
long maxFileNumber(const char* dirName, const char* baseName, const char* fileExt) {
  File dir;
  long maxNum = -1; 
  uint8_t len = strlen(baseName);

  dir = SD.open(dirName, FILE_READ);
  if (!dir || !dir.isDirectory()) return -2;
 
  while (true) {
    char *ext, *name, *ptr;
    dir_t d;
    // Return result if no more entries.
    if (dir.read(&d, 32) != 32 || d.name[0] == 0) return maxNum;
    // Only check existing regular files.
    if (d.name[0] == DIR_NAME_DELETED || !DIR_IS_FILE(&d)) continue;
    // Cast file extension to char* for ease of use
    ext = (char*)d.name + 8;
    ext[3] = 0;
    // Remove any blank fill.
    if ((ptr = strchr(ext, ' '))) *ptr = 0;
    // Skip if wrong file extension. Use case independent compare.
    if (strcasecmp(ext, fileExt)) continue;
    // Cast d.name to char* for ease of use
    name = (char*)d.name; 
    // Don't look at file extension
    name[8] = 0;     
    // Look for baseName with case independent compare.
    if (0 == strncasecmp(name, baseName, len)) {
      // Place for start of number.
      char* bgn = name + len;
      // Iterator
      char* ptr = bgn;
      long n = 0;
      // Get number.
      while (isdigit(*ptr)) n = 10*n + *ptr++ - '0';
      // make sure the end of the file name is a valid number.
      if (ptr != bgn && (*ptr == 0 || *ptr == ' ') && n > maxNum) maxNum = n;
    }
  }
}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);
  if (!SD.begin(chipSelect)) {
    Serial.println(F("SD.begin failed"));
    while(1);
  }
  Serial.print(F("ANALOG,BIN: "));
  Serial.println(maxFileNumber("/", "ANALOG", "BIN"));
  Serial.print(F("DATA,TXT: "));
  Serial.println(maxFileNumber("/", "DATA", "TXT"));
  Serial.print(F("SHORT,TX: "));
  Serial.println(maxFileNumber("/", "SHORT", "TX"));
  Serial.print(F("NOEXT,<null>: "));
  Serial.println(maxFileNumber("/", "NOEXT", ""));
  Serial.print(F("<nobase>,TXT: "));
  Serial.println(maxFileNumber("/", "", "TXT"));
  Serial.print(F("MISSING,TXT: "));
  Serial.println(maxFileNumber("/", "MISSING", "TXT"));
   Serial.print(F("<bad dir>: "));
  Serial.println(maxFileNumber("/BADDIR", "DATA", "TXT"));
 
  Serial.println(F("Done!"));
}
void loop() {}

I ran the program on an SD that had these files root directory:
Quote
12345678.TXT
ANALOG00.BIN
ANALOG01.BIN
ANALOG02.BIN
ANALOG03.BIN
ANALOG04.BIN
ANALOG05.BIN
ANALOG99.TXT
DATA000.TXT
DATA999.TXT
NOEXT00
NOEXT09
SHORT05.TX
TEST.CSV

Here is the test output:
Quote
ANALOG,BIN: 5
DATA,TXT: 999
SHORT,TX: 5
NOEXT,<null>: 9
<nobase>,TXT: 12345678
MISSING,TXT: -1
<bad dir>: -2
Done!

Note: Finding "12345678.TXT" by opening every possible file in the series would take close to forever.
15  Using Arduino / Storage / Re: Read CSV or TXT from SD card into string or array on: April 15, 2014, 06:21:04 am
Here is a simple sketch that reads a CSV file with two numbers on each line.  It doesn't use the dangerous String class.
Code:
#include <SD.h>
File file;

bool readLine(File &f, char* line, size_t maxLen) {
  for (size_t n = 0; n < maxLen; n++) {
    int c = f.read();
    if ( c < 0 && n == 0) return false;  // EOF
    if (c < 0 || c == '\n') {
      line[n] = 0;
      return true;
    }
    line[n] = c;
  }
  return false; // line too long
}

bool readVals(long* v1, long* v2) {
  char line[40], *ptr, *str;
  if (!readLine(file, line, sizeof(line))) {
    return false;  // EOF or too long
  }
  *v1 = strtol(line, &ptr, 10);
  if (ptr == line) return false;  // bad number if equal
  while (*ptr) {
    if (*ptr++ == ',') break;
  }
  *v2 = strtol(ptr, &str, 10);
  return str != ptr;  // true if number found
}

void setup(){
  long x, y;
  Serial.begin(9600);
  if (!SD.begin(SS)) {
    Serial.println("begin error");
    return;
  }
  file = SD.open("TEST.CSV", FILE_READ);
  if (!file) {
    Serial.println("open error");
    return;
  }
  while (readVals(&x, &y)) {
    Serial.print("x: ");
    Serial.println(x);
    Serial.print("y: ");
    Serial.println(y);
    Serial.println();
  }
  Serial.println("Done");
}
void loop() {}
 

Reading this "TEST.CSV" file.
Quote
1,2
34,56
987,-765
1234,-5678

Results in this output
Quote
x: 1
y: 2

x: 34
y: 56

x: 987
y: -765

x: 1234
y: -5678

Done
Pages: [1] 2 3 ... 99