Any Ideas? Write to SD without blocking the processor during the write Process.

Using Arduino Storage
1

Hi folks,

I'm building up a logging solution for the CAN-Bus and i want to capture all sent messages in a raw format to an SD Card.

I'm using the sparkfun CAN shield and an Arduino UNO (but I also ordered a Mega)

The logger works great BUT I can't handle CAN messages during the write process, because the CAN receiver and the SD card communicate over Hardware SPI and interrupting the SD card write to receive a new message messes up the write process. So i have to disable Interrupts (over which the CAN messages are received and put into a buffer) and lose a few messages, resulting in an incomplete Log.

Fat16Lib has already been extremely helpful and constructive towards my problem, but it really looks like doing the write with the current setup won't work. Hooking up a different sd shield, that interfaces over software spi would allow interrupts while writing, but will very likely not be fast enough to keep up with the 500kbit CAN-Bus.

So i went back to the drawing board and tryied to come up with new ideas/work arounds, but maybe my incomplete knowledge doesn't generate all possible solutions, so I was hoping you could throw in some ideas to do the write process in parallel to the CAN receive process.

This is what i've come up with so far:

  • find an SD logger, that doesn't block the ATmel while writing (possibly with an extra IC that writes the data to the SD)
  • connect an SD logger over I²C, so that it doesn't use the Hardware SPI (has anyone done this before? It should be almost as fast as hardware SPI, wouldn't it?!?!)
  • Buffer all data into an external RAM and write the captured data to the SD after finishing the log (problem: 50 seconds of log need 1 MB of raw data - I haven't even seen a RAM for arduino that surpasses 256 Kbit...)
  • USE TWO ARDUINOS - This is the solution that I think about the most at the moment!

There have already been several ideas concerning co-processors
( http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1237871297 , hobby-roboter.de - This website is for sale! - hobby roboter Resources and Information. , Coprocessor )
and I know the basics of multiprocessor scheduling and resource handling, so I think it should definately be possible, to have one Arduino read the Data over his hardware SPI from the CAN controller, put it into a shared RAM buffer (of course using correct lock-outs) and then let the other Arduino fetch that buffer to write it to an SD card.
( sth. like: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1293464810/all )

These are the ideas I have come up, but as i've already mentioned, my "knowledge tool-Box" is limited and I bet you have way better ideas or know instantly, that what I'm considering is complete BS :wink:

I would appreciate any insight/link/idea that you are willing to share with me, because I've already exceeded my capabilities.
Thanks in Advance, Keija

2

I am a still new but, one strategy would be to send "keep alive" messages and not request data, before your write schedule and stop the logging long enough to write to the sd card.

Mark

3

cyclegadget:
I am a still new but, one strategy would be to send "keep alive" messages and not request data, before your write schedule and stop the logging long enough to write to the sd card.

Sadly i don't request the data, it is continuosly sent and the CAN controller only buffers 2 messages, and then overwrites them, when new messages are received (about 3-4 per millisecond and a write takes about 1 millisecond :frowning: ... )

nonethless, thanks for your reply!

4

I worked with other users to make a logger with two Arduinos connected by I2C. I did tests with five byte messages and could send about 1,000 per second to the slave and log them to the SD. That's only 5,000 bytes per second.

It may be possible to increase the I2C speed since Wire uses 100 kHz I2C speed and the avr chip can do up to 400 kHz. The SPI is 8 MHz and software SPI is over 2 MHz so I don't think I2C will work.

Atmel has a product called DataFlash that might work. It is flash memory with two 512 byte SRAM buffers. I bought some samples but never got to them.

I bought SOIC 8-pin parts.

Maybe someone has experience with these.

5

Have you tried putting the CAN bus code inside the interrupt code that fat16lib uses for his 40kHz ADC section?

So, instead of grabbing all that huge amount of data from the ADC, you use that code segment to grab your data from the CAN bus.

fat16lib set that to a 25usec for 40kHz, you could probably increase it to 100usec or more for 4kHz.

Therefore, you guarantee the SD writing, and you tune the interrupt timer to have enough time to complete whatever you do to grab the CAN values. I'm not familiar with CAN, but from what you say it sounds like the device connects to an interrupt and then you have to handle the message. Does it have to respond to every interrupt request? Can you buffer the request to be handled when it's convenient (probably only 10's of usec later). I would think this way you wouldn't need to disable interrupts.

I'm probably not understanding it right, but hey it might give you some ideas anyway.

6

Thanks guys for the input .. after giving alle the options much thought and considering your comments I decided to try my logger with a full size SD first, before getting into fancy solutions (i currently only have a micro SD slot).

nonetheless, thanks and I'll update this as soon as I've tested my logger with the new (ordered) SD card and logger Shield.

...keija

7

You can always get a second SPI bus by bit banging pins.