Ok, ok, now what if you don't happen to be running Windoze or Max OS? And why should you trust that obscure company/organization, when they themselves declare:
www.sdcard.org/legal/index.html:
SD Card Association assumes no responsibility regarding the accuracy of the information that is provided by SD Card Association and use of such information is at the recipient's own risk. SD Card Association provides no assurances that any reported problems may be resolved with the use of any information that SD Card Association provides.
while, on the other hand, they charge $2k to become a member?
Ok, ok, now what if you don't happen to be running Windoze or Max OS? And why should you trust that obscure company/organization, when they themselves declare:
Sadly this is the organization that all the big SD manufactures control and your SD card will be manufactured to their standards.
As I said before, the SD association specifies a specialized version of FAT16/FAT32 and now exFAT for SDXC cards that is optimal for the flash layout and chips used in SD cards.
What I don't understand is why Linux does not have a decent formatter for SD cards.
Linux has tables for handling quirks of USB flash keys and USB hard drives.
If you wish you can use my SdFormatter example in SdFat. It implements the SD associations standard which is specified in a 70 page Confidential document.
The document even specifies the MBR and uses it to control reserved space. You can't just decide on a partition size and location.
Here is an example of one algorithm for FAT12/FAT16 parameters from the standard with proprietary formulas and tables missing.
Annex: Format Parameter Computations
Data Area should be formatted by the following steps.
Sectors per Cluster(SC) is determined from the area size.
Number of Root-directory Entries(RDE) is 512.
Sector Size(SS) is 512.
Reserved Sector Count(RSC) is 1.
Total Sectors(TS) is the number of all sectors of the area.
FAT bits(12[FAT12], 16[FAT16]) is determined by SC and TS.
Sectors per FAT(SF) is computed as following:
(Standard has a proprietary formula)
Number of sectors in the system area(SSA) is computed as following:
(Standard has a proprietary formula)
Number of Sectors in Master Boot Record(NOM) is computed as following:
(Standard has a proprietary formula)
If NOM isn’t equal to BU, NOM is added BU.
Maximum Cluster Number(MAX) is computed as following:
(Standard has a proprietary formula)
Sectors per FAT(SF’) is recalculated as following:
(Standard has a proprietary formula)
If SF’ isn’t equal to SF, SF’ is used as SF. And recalculate from step 8.
If SF’ is equal to SF, parameter computing is complete
Much of the write-up for the SD library is wrong. I wrote the old version of SdFat that is used in SD.h. Too bad the Arduino company has not fixed bugs in the code or the write-up for years.
Having Win XP/32 and Win7/64 I once used external SD formatting tools and always ran into SDinit/read/write issues for Arduinos (Mega, Due) repeatedly.
Now I'm just using standard Windows format (SD 1GB - 16GB, FAT16/FAT32) - and I'm completely fine with them.
I once used external SD formatting tools and always ran into SDinit/read/write issues for Arduinos (Mega, Due) repeatedly.
Yes there are external formatters that are worse than the Windows formatter.
No tool produces a better result than the SD association formatter since it produces the single format agreed to by the companies that make SD cards.
I wrote the base code for the standard Arduino SD library and it is optimized for the standard format published by the SD association. All versions of my SdFat library are tested with the standard format.
So if you use SD.h or SdFat, you should use the SD association formatter.
Since you provided almost no information, no output or hardware description, I can't help much.
SD.h is just a wrapper around an old version of SdFat. The main difference is SD.h runs at a slower SPI rate, 4 MHz, vs 8 MHz for newer versions of SdFat.
Often a hardware/SD combination will run at 4 MHz but fail at 8 MHz.
You can set the set the SPI clock rate to 4 MHz with the SdFat begin() call.
sd.begin(chipSelect, SPI_HALF_SPEED);
Please post complete info about you hardware , SD card model, your code, and output with error info.
effing board restriction to 9000 chars does not allow to post complete code.
Code runs on a DUE at most possible speed for the TFT (ILI9341_due lib, SPI clock up to 84 MHz),
for the SD card I didn't alter anything of the pre-settings.
SD cards are from Transcent, 1GB / 16GB, class 10.
after having maximized the read rate from within my possibilities to hardly 29.477 kByte/s, which is not even half of what I need, I decided to try the SDFormatter tool. Well, my rate dropped down to 28.587, which may not seem much at all, but is a giant loss regarding my needs.
To get a bit more precise, I have exactly 200,000 µs of time to process 45,056 Bits of data and react to some inputs at the same time, while processing means read, iterate over the bits of each read portion (Bytes atm) and write an output pin accordingly, with a precise timing of 4.4 µs each.
So let's change the scale a bit, here's my measuring code:
#include <SPI.h>
#include <SD.h> // modified version
File srcFile;
void setup() {
if (!SD.begin(4, SPI_FULL_SPEED)) {
return;
}
srcFile = SD.open("eight.three");
if (srcFile) {
unsigned long t_start = micros();
for (int i = 0; i < 5632; i++) {
byte bla = srcFile.read();
}
unsigned long t_end = micros();
Serial.begin(250000);
while (!Serial) {
}
Serial.print(t_end - t_start);
srcFile.close();
}
}
void loop() {
}
The output using a "gnome-disks"-fat32-formatted card is: 186588
Same program using the same card, but formatted by SDFormatter4 under Windows (hard inough to manage that ;p ) outputs: 192384
That's a hell of a lot more (recall the 200,000 µs total).
Now here's my actual question: How, if not using a certain formatting tool, will I get this job done in about 100,000 µs?
I actually do need a filesystem btw. I've already had a lot of thoughts about omitting it - finally that's no option. Going raw is a thing I'll definitely try in some other project, though.
Your performance problem is not related to format. Single byte read is very slow and SD access has almost nothing to do with read speed. I am not sure why you see a slight difference in read speed, single byte read should be almost independent of the SD or format.
What version of SD.h are you using that accepts "eight.three" as a file name?
You need to read multiple byte into a buffer. You should be able to get a large increase in read speed, at least a factor of ten.
Look at the bench.ino example in SdFat. It can read at up to 400 KB/sec on an Uno.
Edit:
I ran your example with a correctly formatted SD using SdFat and it printed 76140. Not sure why you get such a long time for single byte read.
I ran bench.ino with a 64 byte buffer and got the following
First, thanks for the reply.
It's he SD library that comes with Arduino 1.6.5. The modification I made is to have it accept an optional parameter for SPI speed:
boolean SDClass::begin(uint8_t csPin, uint8_t spiSpeed = SPI_HALF_SPEED)
"eight.three" is just a replacement for the actual file name, a well known 8.3 styled name that I didn't want to be recognized here. Sorry for that.
The bench.ino indeed output something just above 300 kByte/s reading, didn't have time to take a closer look at it, though.
I'll switch to SdFat and try some project related things with it asap. Funny thing btw, running that exact same code I posted before, only using SdFat.h instead of SD.h, I lose another 26 ms (218,180 µs total). Something's weird here, but I'm on it.
Allthough it no longer belongs here, for
fat16lib:
Your performance problem is not related to format.
I'm gonna post the results as soon as I'll get some.
Funny thing btw, running that exact same code I posted before, only using SdFat.h instead of SD.h, I lose another 26 ms (218,180 µs total). Something's weird here, but I'm on it.
Yes, makes no sense. When I run your code with SdFat I get 76140, almost three time faster. I copied the code and changed SD chip select from 4 to 10 to match my shield.
Edit: I ran the program again and got a longer time. I will investigate more.
Still, the answer for better performance is to do multiple byte reads.
Here is a buffered read that take 16836 micros.
uint8_t buf[64];
unsigned long t_start = micros();
const size_t TO_READ = 5632;
size_t nr;
for (int i = 0; i < TO_READ; i += nr) {
nr = TO_READ - i;
if (nr > sizeof(buf)) nr = sizeof(buf);
size_t ir = srcFile.read(buf, nr);
// byte bla = srcFile.read();
if (ir != nr) {
// read error
return;
}
for (int j = 0; j < nr; j++) {
// process bytes here
}
}
unsigned long t_end = micros();
Funny, but I was having endless issues getting an SD card to work on an ethernet shield. After following every suggestion available on the internet, I finally broke down and bought a different shield. Got the Arduino Ethernet shield 2, and popped in the card. STILL didn't read. Argh... Ran across this thread, reformatted the card with the SD formatter, and it started working immediately. Thankfully, it still didn't work in the other shield, so at least the new shield was worth the purchase.
It says, "Windows : right click on your card’s directory and choose “Format” from the drop down. Make sure you choose FAT as the filesystem."
It's so outdated that it says nothing about FAT32, doesn't say anything about cluster size, and literally says to use the OS formatting. That's what I read long before I found this thread, and it misled me. Someone needs to change it.
I have no choice but to use the OS formatter for my SD cards. There isn't a Linux version of SD Formatter and running it in Wine, it doesn't see the SD card!
There's a license free version of Windows XP for virtual machines you can download for free. For problems like yours I always have a VirtualBox installed with that version.
Might not be the best solution just for formatting SD cards bit at least it is one.
I have no choice but to use the OS formatter for my SD cards. There isn't a Linux version of SD Formatter and running it in Wine, it doesn't see the SD card!
You can download SdFat and use the SdFormatter example on your Arduino.
I designed the SdFormatter example to produce the same layout as the Win/Mac version of the SD Association formatter.