I want to share my library to log records on SD card. I was working on my own project, using Arduino Nano board, and got a requirement to log the process flow to SD card. At this point, my project already consumes 70% of code and 40% of RAM. So when I tried to use standard SD library, project run out of memory.
Then I start looking for any custom and optimized solutions, and found Petit FS by ChaN. But even it does not use sector buffer (saves 512 bytes of RAM), it does not allow to append file.
So finally I reworked code of Petit FS (I removed all extra code, and supported file append).
I also used software SPI, because partial writing does not allow you to share Arduino hardware SPI with some other devices (need to keep CS pin active for SD card till complete 512 bytes block).
Brief parameters of library:
ROM: ~5K
RAM: 73 bytes
Speed: 2.7K per second
Due to custom implementation, library has some limitations, but I think they are not critical. Please check description on github for more details.
If I made a bicycle, and there is a library which better does what I need, please point me, I'll really appreciate.
So far, I hope it will be useful to someone.
If you decide to try it, please share your feedback, thank you!
Thanks very much for posting this. I've only looked at the README, but am curious about how you update used clusters in the FAT - if that is necessary at all. Also a bit puzzled by the 16 deleted empty files.
I skip first 128 clusters, because I skip first sector of FAT (which contains system volume records), and treat all remaining clusters as free (that is why there is a limitation to not store any other files on this SD, because they will be corrupted). So when logger appends file, we know how to fill partially written sector with used clusters, to keep them consistent.
16 deleted files are needed to fill first sector of directory - we get correctly filled file records (even with 'deleted' flag), and filesystem will find our log file. Otherwise there will be null records, which filesystem driver treats as end of directory. And it is not possible to initialize this first sector from library, because we need to partially update data, which is not feasible if we do not use temporary buffer of 512 bytes.
Please excuse my lack of knowledge, but I'm relatively new to Arduino. What do I do with the .cpp file? It isn't referred to in the .h or the .ino. Could you list the steps to get all this compiled and installed in the IDE?
Ok, I've done that. But I don't have an RTC. So I'll need to change the .ino to just write dummy D/T data. Will work on that tomorrow. What I'm interested in is how you deal with marking clusters as used, if you do that, and whether it is necessary to deal with the FAT at all if there is only one file on the SD card.
So I love this idea because of the SD Library eating tons of SRAM. I am getting error code: 6 every time I run the example. I also do not have an RTC so I commented out the define of TINY_SD_LOGGER_RTC. Not sure what's going on or why I cannot get it to work but really really wanting this to!
robertpickett3:
So I love this idea because of the SD Library eating tons of SRAM. I am getting error code: 6 every time I run the example. I also do not have an RTC so I commented out the define of TINY_SD_LOGGER_RTC. Not sure what's going on or why I cannot get it to work but really really wanting this to!
Have you done the card preparation steps listed in the Readme - the 10 empty, deleted files?
Well then I don't know what to suggest. When he posted this last August, I was going to try it, but never got around to it. He hasn't posted here since then.
I really don't understand Nick saying he can append to a file. If a sector is half full, you'd have to read it into a buffer in order to append to it. Unless he never writes partial sectors, in which case there'd be data loss.
Given that the Uno and Nano have 2K of RAM, there's really no need to not have a 512 byte buffer. 1.5K is still plenty to do other things. Coming from a Microchip PIC background I'm astounded at how much RAM gets wasted on Arduinos. I'm currently doing a SD card datalogger app on a PIC with only 1K of RAM and I still have tons left. Programming in assembly, though.
Yes, I know what you mean about ram usage. My first exposure to microcontrollers was the MSP430G2231, which has 128 bytes of ram. That's 128 whole bytes.
And I wrote an SD card bootloader for those chips in assembler that fits into 1K of flash memory and something like 80 bytes of ram. Of course that's just reading, which makes it easier. But I still get that bloatware reaction sometimes when looking at the size of Arduino code.
But what he did is pretty cool. If you're willing to live with some limitations, he demonstrates that you can basically just assign the whole card to the file, and write to it consecutively. So, you know, we don't need no stinkin' clusters and folders. It's just a continuing multi-sector write.
But I agree with you that since the ram is available, it would be nice to set up a couple 512-byte buffers to help preserve sampling speed when the card gets slow, which it will do from time to time.
So I know enough about programming to be dangerous. Lol Part of my problem is that I have a huge number of global variables because I’m creating a laser tag game that will allow setting up game parameters. I would love to learn how to save ram usage. Are you saying it is partially Arduino’s fault?
Compilers really don't know what you're trying to achieve. When you program in assembly you manage everything, which is a blessing as well as a curse.
One thing to experiment with is seeing which variables can do double-duty. If two functions use, say, a float...and those functions never interact, then there's no reason why they can't use the same float.
Hello everyone, I didn't receive notifications about your posts (seems that I forgot to configure them in my profile), sorry for that.
I'll try to answer, if it is not too late
How can you possibly append to a file's data area if you don't buffer a sector?
This is from readme file: "TinySDLogger uses software SPI, because partial writing approach requires hold of CS pin till write 512 bytes". I do not release CS pin, thus transaction is not committed, and it is in SD card internal buffer. Obviously, I can't append to sector after device restart, in this case I start append file from next sector, as result "Close file method fills remaining bytes (to round up to 512 bytes) with spaces and line feed in the end. This may result in gaps between log sessions".
I am getting error code: 6 every time I run the example
It means that SD card is not FAT32 format. Can you say details about your SD card? Like size, manufacturer. And if you can read first 512 bytes of your card by some tool like Active Disc Editor, put it here.
But I agree with you that since the ram is available, it would be nice to set up a couple 512-byte buffers to help preserve sampling speed when the card gets slow, which it will do from time to time.
This library designed for projects where you do not have free 512 bytes (as in my case, you need to add logger facility to already existing project). If you have enough RAM, it is obvious to use standard SD libraries, which gives you much more functionality.
Regarding 'card gets slow', I think it is not a case for sequential writing, when we do not have any scan loop to find a file or next free block, or random read.