Hello, I am writing a program to find out how fast data is being written to a SD card. If I remove all unnecessary statements (such as a if-statment to check if certain number of lines has reached), how do I savely close the file and end the program before I remove the card?
file.flush(); // empty buffer
delay(1000);
file.close();
Thanks. How do I integrate these to within the loop() without throttling the sampling in any way?
"file." is the name you give the SD card instance, usually when you open the file (so, maybe, use another descriptive name to avoid confusion... like "bqFile".
- open bqFile in
setup() - read/write to bqFile in
loop()
When finished read/write-ing (in loop())
- (flush bqFile)
- close bqFile
Thanks. I know this procedure but the problem is the "When finished read/write-ing" part. I think I was told to do the sampling as fast as possible so I guess I need to avoid the if-statement within the loop() function to test when, say a certain number of lines have been written or when a button/key is pressed. If I just unplug the card, I might corrup the file/file system or damage the sd card. Without using flush() to clear the buffer and then closing the file, the data file may not be written successfully.
Please refer to Post#31 and Post#47
Before getting to excited about throttling the sampling, consider the data for the SD card is written in 512 byte blocks. You can sample as fast as you can until you hit the 512 byte limit and then the rest of your code, including the sampling is blocked from executing until the 512 byte block is written and the file header is updated and written to the SD card. Only then will your code continue. Facts of using the SD card. It all began in the dark ages with the floppy disk in the PC.
Question is: Why the SD card.
The ESP32S3 has built-in 8MB or 16MB flash and BT+WiFi capabilities.
You can log a lot of data on that flash before offloading with WiFi or BT.
If you are not going to use WiFi or that flash, then why did you buy an ESP32S3.
A RTC is also not needed with WiFi near.
Leo..
So to avoid this problem, just created a large arrary to hold the data and then write them to the SD card or offloading with WiFi or BT all at once at the end?
There is no specific reason for using SD card only except that I know how to do it. By using the "biilt-in 8MB or 16MB flash", do you mean creating a big array to store to data? Or there is a special way to store the data in the flash perhaps for faster access than using an array?
By "RTC is also not needed with WiFi near". Do you mean just use NTP regularly to get the clock from the internet?
Yes, but design the array entries so as to use the fewest number of bytes to store your data. At some point you can learn to compute the maximum number of entries that will fit in the remaining memory because as you develop and modify the program that size will probably shrink.
Yes, default NTP updates are 3-hourly, which is more than enough to keep time within one or two seconds. NTP is built into the ESP core and only requires one line in setup.
There are plenty of LittleFS data logging examples online to study.
Leo..
This statement will only require a matter of microseconds to execute. So it is not something you need to worry about at this point. In addition, it is possible this type of check could actually be utilized to improve the sampling rate by the technique of staging your samples in an array in RAM, then only undertaking the significantly time consuming process of writing that data to the SD card after the sampling session is complete, as you described in post #8.
A digitalRead call does take more time compared to a simple value check. For the purpose of the simple test to get an idea about the maximum possible sampling rate, I would recommend just taking a fixed number of samples, as is done by your existing code:
(the variable names are misleading as this is actually the sample count as implemented in the code)