Show Posts
Pages: 1 2 3 [4] 5 6 ... 99
46  Using Arduino / Storage / Re: sdfatlib: Possible to work around or minimize high-latency reads? on: March 19, 2014, 10:07:13 pm
Does your display interface use the SPI bus?

You can't use SPI in an ISR while a program like SdFat is doing a transfer in non-interrupt code.
47  Using Arduino / Storage / Re: sdfatlib: Possible to work around or minimize high-latency reads? on: March 19, 2014, 03:33:38 pm
You could use the same kind of buffering that audio players use.

I wrote a wave file player about five years ago that plays 16-bit 44.1 ksps wave files http://code.google.com/p/wavehc/downloads/list.  This player delivers data to an DAC at a constant rate of 88.2 KB/sec using a 328 Arduino.

This player uses a timer1 IRS to send data from buffers to the DAC.  Data is read from the SD into buffers in non-interrupt code.

Define your needs as pito suggested.

What is the data rate and how is the data sent to your POV display?  Give details of the interface to the display.
48  Using Arduino / Storage / Re: Does SerialPort library work with the Atmel Studio VisualMicro plug in? on: March 19, 2014, 01:03:26 pm
Looks like you are allocating too much buffer for your serial ports.
Code:
SerialPort<0, 4096, 0> NewSerial;
SerialPort<1, 4096, 0> NewSerial1;
SerialPort<2, 4096, 0> NewSerial2;

This is 3*4096 or 12288 bytes.  The Mega only has 8 KB of SRAM so I think you are just having repeated crashes due to the stack overlapping serial buffers.  This explains the delay(1600) problem.
49  Using Arduino / Storage / Re: sdfatlib: Possible to work around or minimize high-latency reads? on: March 19, 2014, 12:40:12 pm
You can't avoid long read latencies.

SD cards are block devices with 512 byte blocks and an entire block must be read.  SdFat has an internal 512 byte cache so your low latency reads are just the time to copy data from the cache.  Longer latency reads fill the cache by reading a 512 byte block.

Your longest latency reads occur when SdFat must locate the next cluster.  This requires reading a 512 byte FAT block in addition to the data block.  If you SD card is properly formatted with 32 KB clusters, this will occur on 32 KB boundaries.

It is possible to avoid the FAT access by using raw reads of contiguous files.  The requires lots of memory and tricky programming.  You will still have latencies of at least 1000 usec when you read a block.  You may still see some 2000 usec or greater latencies depending on your SD card. 
50  Using Arduino / Storage / Re: Stack address > 2k for ATMEGA328p? on: March 18, 2014, 06:41:12 pm
The first 256 bytes of address space are the 32 general registers and various I/O registers.  SRAM starts at 256 decimal.

The stack starts at 2303 = 256 + 2047.  The stack pointer is decremented when items are pushed.

Looks like freeMemory() has not been fixed for the latest change to malloc./heap.  The Arduino company changed malloc() to a nonstandard version then changed it again.  This broke lots of free memory programs twice.
51  Using Arduino / Storage / Re: interfacing multiple sd cards. on: March 18, 2014, 04:21:12 pm
SdFat supports multiple cards.  See the two card example. 

Each card requires a 512 byte block buffer so it's nt very practical on an Uno but works well on a Mega.
52  Using Arduino / Storage / Re: Help with SD FAT raw write to SD card for fast DAQ with external interrupt. on: March 18, 2014, 07:48:24 am
I have started development of a new binary logger for analog pins.  Results on an Uno are encouraging.

Here is a debug test logging 5 analog pins every 200 usec to a $5.00 4 GB SanDisk card.  This is 25,000 ADC values/sec.
Quote
Sample pins: 0 1 2 3 4
ADC clock MHz: 0.500
Sample interval usec: 200.00
Sample Rate: 5000.00
Deleting old file
Creating new file
Erasing all data
Logging - type any character to stop
Truncating file
Max block write usec: 920
Record time ms: 1324513
Sample count: 6622559
Samples/sec: 5000.00
Overruns: 0
Done
I ran this test for about 22 minutes and recorded over 33 million ADC values.  The maximum time to write a 512 byte block is 920 usec.  This is a bit longer than the 720 usec in the above tests due to ISR time for the ADC. Each block holds 10 ms of data.  More testing is required to verify that the logger can run on an Uno for hours at this rate.

The ADC clock rate is 500 kHz so accuracy is slightly degraded.  See the attached chart.

I am developing a C++ PC program to read these binary files.

Edit:  The PC program converts the 66 MB binary file to a 112 MB csv file in about 10 seconds.  The PC is fast with ssd drives.
53  Using Arduino / Storage / Re: Cant get Micro SD reader to initialise! on: March 17, 2014, 11:43:24 am
Your problem may be a wiring error.

SO on the module should be connected to Arduino pin 50, MISO, Master In Slave Out.

SI on the module should be connected to Arduino pin 51, MOSI, Master Out Slave in.

You say this is how they are connected.
Quote
SO 51
SI 50
54  Using Arduino / Storage / Re: SdFat: counting number of files in directory? on: March 15, 2014, 10:40:53 am
No.

Since used directory entries are randomly scattered through the directory file you can't seek to the nth used entry.

openNext() just sequentially reads the directory file.
55  Using Arduino / Storage / Re: Help with SD FAT raw write to SD card for fast DAQ with external interrupt. on: March 15, 2014, 10:37:20 am
How about 700 KB/sec without latency problems on an Uno.

 I found an old 1 GB SanDisk card that runs flat out with a max write latency of 720 usec for a 512 byte block.

Looks like I should write a new fast ADC logger example using multi-block streaming.

Quote
Start raw write of 51200000 bytes
Elapsed time: 71.547 seconds
Write rate: 715.6 KB/sec
Max write time: 720 micros

This card was manufactured in 2006.
Quote
Card type: SD2

Manufacturer ID: 0X3
OEM ID: SD
Product: SD01G
Version: 8.0
Serial number: 4217130864
Manufacturing date: 12/2006

cardSize: 1015.81 MB (MB = 1,000,000 bytes)
flashEraseSize: 32 blocks
eraseSingleBlock: true
56  Using Arduino / Storage / Re: SdFat: counting number of files in directory? on: March 15, 2014, 10:14:36 am
By random order I mean files are created using the first free directory entry.  This means you can't know where a new file will be created.  Deleting a file causes the directory entry to be free.

You can use openNext() to count the files.

To open the nth file just skip files using openNext() until the nth file.
57  Using Arduino / Storage / Re: SdFat: counting number of files in directory? on: March 15, 2014, 09:27:49 am
This is possible but not very useful since the order of files in a FAT16/FAT32 directory is random.

Why do want to do this?
58  Using Arduino / Storage / Re: Help with SD FAT raw write to SD card for fast DAQ with external interrupt. on: March 15, 2014, 08:09:21 am
Looks like the cheap, $5.00, card could log data at 100 KB/sec without latency problems on an Uno.

Here are some initial results that show promise for a very fast logger on Uno with cheap cards.

Here is a short 500 second test writing 50 MB.
Quote
Start raw write of 51200000 bytes at
102400 bytes per second
Please wait 500 seconds
Done
Elapsed time: 500.000 seconds
Max write time: 1108 micros
Overruns: 0
Longer tests are needed but two 512 byte buffers can easily handle this max latency.

Looks like other cheap cards can be used at high rates also.  200 KB/sec may be possible with more buffer on a Mega or 1284P.

I discovered that the latency pattern for multi-block transfers is bad on cheap cards unless you delay between the command to prepare for a transfer and sending the first block.

Here is the result for attempting to write at 200 KB/sec with no delay on the $5.00 card.
Quote
Start raw write of 51200000 bytes at
204800 bytes per second
Please wait 250 seconds
Done
Elapsed time: 250.143 seconds
Max write time: 65208 micros
Overruns: 19
fileBlock,micros
65,2424
80,41884
81,65208
82,27764
5569,3612
8257,3276
11009,2976
13761,2896
16449,3188
32833,2596
49217,3168
54721,2500
60161,2820
68353,3424
71105,2608
73792,3628
79297,3224
95681,3280
98369,2704

The card can't keep up in the beginning but settles to a max latency of under 4 ms.

If I delay one second before sending the first block, the card has a latency under 4 ms.
Quote
Start raw write of 51200000 bytes at
204800 bytes per second
Please wait 250 seconds
Done
Elapsed time: 250.017 seconds
Max write time: 3912 micros
Overruns: 19
fileBlock,micros
5569,3232
8257,2428
11009,3328
13761,3228
19201,2456
27393,2872
32833,3012
41025,3128
46529,2344
51969,2148
54720,3172
54721,3912
60161,2996
62913,3524
76545,3608
79297,2292
84737,2468
92929,3732
98369,3184
Overruns would happen on an Uno with two buffers but 4 ms at 200 KB/sec is 800 bytes so more buffers on a Mega or 1284P may work.

The next problem would be to acquire data at 200 KB/sec.  This is one 512 byte block every 2.5 ms.  On average 800 usec is required to transfer the block to the SD so that leaves less than 1.7 ms to acquire 512 bytes of data.  Probably not possible.
59  Using Arduino / Storage / Re: SD card read returns 0xFF on: March 14, 2014, 08:25:19 pm
I believe the Arduino Ethernet is 328 based and has 2 K SRAM.

It appears that this:
Code:
       Serial.println(F("FR:"));
        Serial.println(freeMemory(), DEC);

Prints this:
Quote
FR:
1657

This must be wrong since the SD library alone has a 512 byte block buffer.

I suspect there is a memory use problem.
60  Using Arduino / Storage / Re: Help with SD FAT raw write to SD card for fast DAQ with external interrupt. on: March 14, 2014, 02:37:51 pm
Quote
I see, so my current understanding is the most feasible approach (most universal one) covering vast majority of sdcards available (ie even cheapo sdcards without any pre-selection) would be a standard SdFat API with NilRtos and its FiFO,  supported by a larger RAM (ie 1284p) for faster data streams (25-100kB/sec).

This is best for most people.

I should write a good example for raw write since it works well and is the mode SD cards are designed to used.

I just ran a test on a card in this mode.
Quote
Start raw write of 51200000 bytes at
25600 bytes per second
Please wait 2000 seconds
Done
Elapsed time: 2000.000 seconds
Max write time: 720 micros
Overruns: 0

This is an ATP 1 GB industrial card.  It never took longer than 720 usec to write a 512 byte block at the rate that markallmatt wants to log data so two 512 byte buffers on a Uno would work fine.

Probably a low cost card would work fine at 25 KB/sec.

Edit: I did a test with a $5.00 4 GB class 4 SanDisk card and it would be fine for 25 KB/sec.  Probably could do 50 KB/sec or more in an Uno with two buffers.

Edit: I did a short 1000 second test of the cheap SanDisk card at 50 KB/sec.  Amazing for a $5.00 card.
Quote
Start raw write of 51200000 bytes at
51200 bytes per second
Please wait 1000 seconds
Done
Elapsed time: 1000.000 seconds
Max write time: 724 micros
Overruns: 0
Max write time for a 512 byte block is 724 usec so a block is transferred at 707 KB/sec.
Pages: 1 2 3 [4] 5 6 ... 99