I'm currently working on my 4th year project and I've run into a speed bump. Part of my project is to log the incoming sensor data on an SD card using a shield. I'm using an Arduino Uno R3 with the SD shield 3.0.
I've managed to store data, but the problem is that I'm not getting nearly enough data points when I check the SD card. I connected my Arduino/shield to a wave generator and generated a simple sine wave. There are a lot of 0's and few points.
I've tried changing the baud rate, the ADC, etc, and nothing has worked.
Attached to the post is the code that I'm using as well as the Serial Monitor Output. Hopefully someone out there has some insight to my problem!
You are using the SD library, which is outdated version of the sdfat library that is up to date. So you are not getting very fast speed with the SD library. Say if you store your data in SRAM and then dump all data to SD card, you can, with an UNO, possibly store up to 1,000 data points in SRAM if you use 8-bit ADC resolution (either read and divide by 4 or set the ADC to 8-bit explicitly). Then you write your data to SD card file at the end. So have a button to start storing data in SRAM. Once full, dump to SD card. Each SD card file will take 512 bytes SRAM as buffer so your 2K SRAM will only have less than 1.5KB left to begin with. That is enough to store 1,000 8-bit numbers, at the maximal ADC rate or lower.
Oh ok, that makes sense. I'm using the code provided by this website for datalogging, I've pasted the link below.
How would I go about storing the data in SRAM? This is my first project where my part is mostly software and I'm not that strong just yet. Any suggestions?
It does up to 40,000 SPS and logs them to an SD card as a BIN file which can be converted to an EXCEL CSV file one of two ways. Easiest way is to select "Convert" from the serial monitor prompts after stopping the logging and wait while it does the conversion. This can only be done immediately after stopping the logging. The other way is to use a command line prompt by reading the instructions in the README file.
I prefer the former. You can see my plots (plotted in Excel) in Reply#14 (40000 samples per second).
Read my posts about the compiler errors I got because I didn't know the BIN to CSV convert program CANNOT be in the LIBRARY folder. (it must be moved outside that folder).
FYI, you will never get fast logging using serial prints (in realtime) . It is too slow.
I see, I have a question. My project is an ECG monitor so along with saving the analog data to an SD card, the other part is sending that data using Bluetooth to an android phone.
Is there any way I can do the conversion in order to save to the SD card AND continuously send the input to the android phone without having to stop logging data? Kinda defeats the purpose of the project.
Not with a 16-bit uP. I don't know what the maximum data transfer rate is for bluetooth but you can't convert BIN files to usable data in realtime so your only option is finding a uP that has the speed to log data AND transmit at the sample speed you need. What is your maximum logging rate now ?
At this point I haven't used a standard rate, I was playing around with different speeds but I didn't know it wasn't possible at the time. At this point I'm using a baud rate of 9600.
Is there a uP that you can suggest that does the job instead of an Arduino? My SD and Bluetooth shield are Arduino compatible so they kind of all go together.
You can't isolate the bottleneck until you try with 115200 baud. Then you are maxed out on communications and you can only increase your logging rate. At some point , increasing your logging rate
results in no improvement because the bottleneck is the baud rate.
You should look into ways to transmit the data to the bluetooth in realtime instead of writing it to an SD card. The SD write time is a bottleneck + the read time.
We`re using the SD card to save the data as well as transmitting the information over Bluetooth. So the SD card is a backup in case anything happens to the device itself.
I've changed the sampling rate, I found that site a couple weeks ago and adjusted it and increased the baud rate to 115200. I altered the code to include the new ADC clock code as well as my SD code together and it didn't seem to change anything.
What's your sample rate ? (use a counter to count samples and print "START" & "FINISH" after 10000 points. Use "millis()" to get the start time and finish time and print that.
We made the sampling rate at the highest it can go with the preset values of the ADC.
We divided by 16 instead of the standard 64 and did not see a difference. This was also done attempting to write to the SD card through the Arduino so with the outdated SD library it could make the difference.
If I updated the SD library and tried with the data logger example you provided, do you think that would make a difference?
You haven't posted any data. (of value). You need to do the tests I suggested to find out
what the total time is from start to finish for 10000 points using millis(). Without a baseline , you have no way to measure the improvement if you make any changes. At this point, without that information, it's impossible to answer your question. You can use battery backed ram to eliminate the SD card write bottleneck and check the improvement. Then you can explore other ways to transmit the data but without the baseline you're spinning your wheels.
