Hi all, I am working on a project that requires reading files from an sd card and then using the information inside the main part of the program (void loop). The problem I am facing is the size of the variables I need to store. (STM32F103 with 20KB RAM and 128KB flash memory). Is there a way after reading the values from the sd card to store them inside the flash memory, I am not changing them, only as reference values for later. Thank you, any comments would mean a lot!
This library will probably help.
Thank you!
@red_car Have you ever used PROGMEM, I found that this also saves data in Flash memory. Do you have experience working with the library I need to store an array of 1000 strings, and later send specific values on the serial port.
What are you actually trying to do? Are the strings all the same length?
Nope, the length of the strings is actually varying value, but not longer than 15 chars, I know I could use char[1000][15] which will allocate some smaller amount of memory, but this is also not enough.
= 128,000 bytes.
= 15,000 bytes. Not sure I understand why you think you have storage concern?
To the moment I was using the RAM memory, and also have some 5 more arrays of char and uint8_t with 1000 elements. Witch clogs the 20KB of RAM. I am also worrying that the flash memory has a life cycle. The values should be stored in every start-up. I will try to give more over on the problem in the next reply, so you could see the problem
That would help... understanding the problem is very important... and then (only then), we can think about possible solutions.
I have the following case, I am opening a txt file from an SD card containing messages (every message has an id). I need to store all the information of the message inside the program because I am using them later. The loop is waiting for a message-id from the serial port, when I get one, the program is searching for the id and then sending the corresponding message back through a serial port. The messages are stored once in setup and are never changed. The program works perfectly and fast with everything that can fit inside the RAM. Thank you for everything
Maybe.
You could store them in the flash memory... it will be a lot easier if all the messages are the same length.
But, it is also possible just to leave them on the SD card... open the file, and then move the read head to point to the message you want. Also needs the messages to be the same length.
fseek is a C function that allows you to move the read location within an open input stream.
I see the point using fseek, but the messages are not the same length so I don't know where to jump, and is impossible to calculate the jump (size in bytes).
Here is the same problem as above...
I am thinking of creating in setup really small files with id as name, so for every match to open the file and send it. Did you how long would it take to open a file of 50B to be opened?
Can you make them the same length? Even if they are not the same you can just pad them to the same length with spaces.
Opening/closing files is quite expensive time-wise.
If there is no way to make the messages the same length in the file then you just need some kind of index that you load at the start that provides the storage location of each message (either in the flash or in the file).
There is one more problem with using the Flash memory, from the information provided it has a lifecycle of 10000 cycles, which for storing values that are not changed every start-up (calibrating sensors, constants) is okay, but in my case, I need to store the values on every start-up and the program could store new values every time.
You can easily test that yourself.
If that happens 3 times a day, you have 3000 days before you will reach those 10000 write cycles. And if you implement wear leveling it will be a lot longer.
How often is that?
You can consider to add a FRAM module if necessary; like EEPROM but with nearly unlimited number of writes.
If it is a unique ID you can use the ID as an index into an indexed sequential file.
Have tested (~1000 using SD, ~83 using SdFat), really slow compared to under 50ms, which is recommended for my use case.
This should be more often, about 50 times a day
That could be a solution, but I will try to find something software related because right now I am limited on hardware also.
Right now, I am trying to use SQLite lib, in which I found really impressive times for retrieving data. Thank you all @sterretje @anon57585045 @red_car for commenting, any ideas are welcomed.