Arduino Solid-state storage

Projects General Guidance
1

Hello,

I’m after some guidance here, I’ve made a model rocket data logger as per the picture. Works great. Stores the data from the flight onto an SD card for use after.

However, I’m looking to upscale this project and would like to use solid state memory as the vibrations during a launch could corrupt the data storage due to the pins touching the SD card coming loose or momentarily disconnecting.

I’m currently creating a .csv file and storing strings on it, each line being one sample of all the data points.

My idea is to store the same data to either EEPROM or SPI Flash then after the flight once it’s Landed transfer all the data to the SD.

I’ve made a program using EEPROM that will store each byte of data separately onto it but that seems a clunky approach. Could I make the .csv in the flash chip then somehow copy it over as one, or line by line into the SD card afterwards?

Thanks

IMG_2439
IMG_2439768×1024 75.7 KB

2

You may consider FRAM instead of EEPROM.
Is faster and non-volatile.

how much data do you write during flight?

3

Which Arduino board are you using ?

4

Adafruit sell a nice SD flash card

1 Like
5

The EEPROM should be fine unless you have problems with not being able to write fast-enough or if you're running out of memory.

It's not volatile, and of course it's solid state as-is an SD card.

And I assume you don't want to add any unnecessary weight. And adding extra electronics means more connectors/connections which are vulnerable to shock & vibration.

6

I don’t know much about FRAM, can you write strings to it in one go or do they need to be broken down and sent byte by byte?

Also when it comes to transferring the data from the FRAM to the SD are there any gotchas with trying to separate out all the different strings?

So each string is 40 bytes or so, and looking at taking no more than 1000 samples. That order of magnitude.

7

So I appreciate SD is solid state but there is a physical spring connection to the SD which is vulnerable to vibration. Whereas a chip EEPROM/FRAM etc is soldered so won’t suffer the same under high vibration.

8

There are no mechanical connections on the Adafruit module, it is basically a very large FRAM.

EDIT it comes preformatted for a file system

1 Like
9

Pro-micro in the picture but I’m planning to use a nano every for the next one.

10

How much data do you need to store and why are you storing strings ?

11

I’ve just found it, that might be precisely what I’m looking for. I’ll do some research and see how easy it is to transfer a .csv from one device to another.

Thanks!

12

40 bytes per string and max 1000 samples. Currently it’s doing a lot less but I want to future proof it.

I’m writing strings to a .csv file on the SD card currently.

13

how large does the storage area needs to be?

in the end the data is serialised in most cases, you just might not see it depending on the API/library

14

With high speed data logging, which I guess is what you would need for model rocketry, it would quicker to store the raw data rather than convert it into nice human readable text on the fly.

You could the dump out the raw data as a CSV file later on.

You need to decide what data you are going to record for each sample. I guess you already have this. Then work out how many bytes that is gong to take (less if you record raw data rather than ASCII text). You can then multiply that by the number of samples per second to get the number of bytes per second. Then multiply that by the flight time and add in some pre-flight time period and post-flight time period. That will give you the approximate storage requirements for 1 flight.

That may then steer you to towards a particular storage medium - FRAM, FLASH, EERPOM etc.

1 Like
15

@robtillaart
Hi
I'm interested in your library. This is the first library for FRAM/EEPROM, which takes into account the difference in addressing of small and large chips. I would like to ask a couple of questions, but you don’t have a discussion section on Github.

Maybe you have a separate thread on your library on this forum?

16

@b707
Please open an issue on Github, works perfect for questions.

17

So having been doing a lot of reading around this I’ve narrowed it down to 2 options;
EEPROM & Flash

EEPROM would be easy, no conflict between the i2c protocol and Serial of the SD card. My only concern is for this application is would it be enough memory, it would be nice to future proof it a little in case I wanted to add more data or longer sampling.

For my project I plan to use 21 bytes of data, samples at 20Hz for the first 20 seconds then sampling at 5Hz for the next 3 minutes.

21x20x20 + 21x5x180 = 27kBytes

Looks like: time (mS), alt (m), vertical speed (m/s), lat, long, flight phase

Unsigned Long, float, float, float, float, byte.

Well within i2c EEPROM size, but could be easily breached if the project were to grow/change etc

So

Flash seems to come in the standard external flash variety and the SD card version that someone linked above.
I’ve read about using two SD cards on the same device and having different CS pins to determine which card you’re using.

Q: Would I be better off writing my 21bytes of data as a string, then retrieving it one at a time, or writing the data as 5 floats and a byte then block reading 21bytes of data at a time to re-write onto the SD card? As the address can just be revised by 21 each time.

My guess is that storing it as a string, the size will change slightly depending how many characters are being used. Which means I’d have to store the address for each entry somewhere while data logging, or could I store this in the flash as well?

Thanks!

18

In my opinion, it's not the job of the data logger to log data as a nice human readable string. That's fine if you were logging infrequent events, but for continuous logging I would use raw data that takes as little time as possible to write to your chosen type of storage.

I just had a look and Farnell UK are showing various 512k x 8bit EEPROMS with an SPI interface from Microchip & ST Micro. That should give you plenty of room for expansion.

19

Would there be any deconfliction required with using eeprom on SPI and an SD card on the serial port?

I think I’m right in saying I couldn’t use them at the same time so would I have to read say 420 bytes of data into a buffer, then write the buffer on the SD card, close and repeat?

20

I'm not sure what you mean by this. SD cards and the AdaFruit SD Flash card (link in post #4) use the SPI bus.

You can have more than 1 device on the SPI bus - each device has its own Chip Select pin to indicate which device the host micro is wanting to communicate with.