next topic »

[on:]

I am currently working on a data logging project using the Arduino Mega and the CAN-bus Shield from Sparkfun (http://www.sparkfun.com/products/10039) with the SdFat libraries. But now I am at a point where I am looking for ways to up my sample rate.

I found the Chipstobits Storage Shield (http://www.chipstobits.com/blog/storage-shield/) and it seems like a perfect solution by integrating another micro controller to handle all the SD formatting and writing and freeing up Arduino resources.

The only reason I haven't purchased this shield yet is because I have questions about it and none of the contact points on the website have provided any response. I emailed the two email addresses that I could find on the website last week but have still not received a response. I also tried to call Steven Gifford's (creator) phone number which he posted on his website, but either got a busy signal or straight to voice mail every time.

Has anyone used this shield? Specifically has anyone used this shield with the Mega? What kind of write speeds did you achieve? What communication protocol were you using? Has anyone used hardware SPI with this shield and the Mega?

[Reply #1 on:]

But now I am at a point where I am looking for ways to up my sample rate.

have you seen - http://arduino.cc/forum/index.php/topic,64314.0.html - for fast storage? could be an alternative?

[Reply #2 on:]

I did see that and the subject got me excited, but I'm not sure that it is relevant to my project.

We are using one of the Mega's serial lines to read values rather than analog pins (they use Analog 0 to get these alleged 40000 samples per second). We are are also using a 10-bit ADC. Would either of these factors affect the effectiveness of this new SdFat library and the methods in fat16lib's post?

[Reply #3 on:]

He is now using a 12-bit or 13-bit ADC over SPI.

Considering you are going over serial & not getting many samples/second, you should have plenty of headroom to do the SD writing with sdfat.

You should also read the thread that came before the 40ksps one:
http://arduino.cc/forum/index.php/topic,64085.0.html

[Reply #4 on:]

The Arduino 10-bit ADC is very slow, 115 usec per read.  I posted a fairly fast data logger using the built-in ADC here http://code.google.com/p/beta-lib/downloads/list  It does 2,000 samples/sec to good quality SD cards on a 328 Arduino and up to 4000 on a Mega with larger buffers.  This logger produces text files.

I will soon post a binary data logger that uses a faster SPI ADC and can log up to 40,000 12-bit samples/sec.  It requires a high quality SD card like the SanDisk Extreme.  The very low write latency of these card is crucial, not the speed.

[Reply #5 on:]

I am logging information received on the Serial lines of the arduino. So I don't think that example pertains to my project because it is interrupt driven and if I am not mistaken, you can't have an interrupt based on Serial receive because the Serial libraries already utilize interrupts.

[Reply #6 on:]

I believe fat16lib is using SPI (doesn't even need hardware SPI) for SD writing in the normal fashion (ie loop() section).

There's no reason you can't use the sdfat library for your serial sensor.

He is using the interrupt to get accurately, evenly spaced samples, and writes to the SD card when not actively capturing a sample in the interrupt code.

I don't think it's out-of-the-question to put your Serial commands in a very slow timer-triggered interrupt either.

At least that's how I understand it. I could have it wrong.

[Reply #7 on:]

On a Mega you should write 512 byte blocks.  Allocate a 512 byte buffer and fill it by using Serial.read().  Then do 512 byte writes.  SdFat will write your data to the SD without a copy if you write only 512 byte blocks so your writes are aligned on block boundaries.

To prevent data loss during writes, make the Serial ring buffer as large as possible by editing HardwareSerial.cpp.

Code:

 #define RX_BUFFER_SIZE 128

Replace the 128 with a larger number if you have RAM to spare.

SD cards have occasional write latency times of 150 ms in the mode SdFat uses to write files. You need a large ring buffer to avoid data loss.  At 115200 baud you may need almost a 2KB ring buffer.

I use special multiple block raw write commands and SanDisk Extreme cards in my binary logger to avoid the latency problem.

[Reply #8 on:]

I appreciate your feedback fat16lib, and you are doing some really interesting stuff.

But my original questions about the Chipstobits Storage Shield still remain unanswered. I am very interested in the shield but it seems the more I look through the documentation the more questions I have.

1)Can you reroute the SPI pins on the shield to the equivalent pins on the Mega in order to use hardware SPI with this shield and the Mega?
2)I downloaded the documentation and library and have been looking through it to try and answer some of these questions I have about the shield and I came across this:

Code:

#include <avr/pgmspace.h>


/*******************************************************************************//**
 * \file spiMaster.cpp
 * \author http://www.chipstobits.com   - Steven Gifford
 * \author Copyright 2009-2010  Chips To Bits
 * \author Licensed under Creative Commons 3.0 Attribution, Share Alike
 *
 * Low level functions to support hardware level  master SPI communications.
 ******************************************************************************/

    
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
 
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif

// pin number on the port (not the chip)
#define MISO_PIN 4
#define MOSI_PIN 3
#define CLK_PIN  5
#define CS_PIN   2

// Data Direction port
#define MISO_PIN_PORT  PORTB
#define MOSI_PIN_PORT  PORTB
#define CLK_PIN_PORT   PORTB
#define CS_PORT        PORTB


// Data Direction Register
#define MISO_PIN_DDR   DDRB
#define MOSI_PIN_DDR   DDRB
#define CLK_PIN_DDR    DDRB
#define CS_DDR         DDRB

#define SELECT()        cbi( CS_PORT,       CS_PIN ) 
#define DESELECT()      sbi( CS_PORT,       CS_PIN )

On what board are the MISO, MOSI, and CLK on pins 3, 4, and 5? I know on the Uno they are 11-13 and on the Mega they are 50-52. What am I missing?

3)Does anyone have any idea how to get in contact with anyone at this website or under this Chipstobits organization?

[Reply #9 on:]

For the Mega:

Code:

#define MISO_PIN 3
#define MOSI_PIN 2
#define CLK_PIN  1
#define CS_PIN   0

These are the bit numbers in Port B, nothing to do with Arduino pin numbers.
See Table 13-6 in this http://www.atmel.com/dyn/resources/prod_documents/doc2549.pdf.

[Reply #10 on:]

Have you looked at the OpenLog data logger from Sparkfun? Only uses a serial port and writes to a microSD card. I been using one for about 6 months logging data from a Mega thru serial port 3 with no issues. Logs data as fast as I can send it at 19200 baud.