I'm working on a project that drives my LedStrip. But as a test I wanted to write a small script to read data from an micro SD card. The ParseInt() function seems to read in all the values pretty quickly for a small test set. So I expanded to more values, but than the SD card would not initialize anymore..
This is my script. The 1LST file is a '1 line space test' file with the numbers 0 - 899 separated by spaces. When I first tried the uint8_t declaration of x[900] it did not work, but it worked for uint8_t x[800]. However it would then only print values between 0 and 255, which makes sense, so I wanted to change it to uint16_t x[900].
The IDE did not give an error wrt the memory usage, but after running the script, the Serial monitor gave an output that reads:
It seems to be some kind of memory issue, but I dont see why that would stop the SD card from being initialized..
Is there anything I can do to fix this?
#include <SPI.h>
#include <SD.h>
File myFile;
void setup() {
// Open serial communications and wait for port to open:
Serial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for native USB port only
}
Serial.print("Initializing SD card...");
if (!SD.begin(10)) {
Serial.println("initialization failed!");
while (1);
}
Serial.println("initialization done.");
// open the file for reading:
myFile = SD.open("1LST.txt");
if (myFile) {
Serial.println("1LST.txt");
int Size = 900;
while (myFile.available()) {
Serial.println("should be printing now");
uint8_t x[Size];
// uint16_t y[Size];
for(int i = 0; i < Size; i++){
x[i] = myFile.parseInt();
}
for(int i = 0; i < Size; i++){
Serial.println(x[i]);
}
myFile.close();
}
} else {
// if the file didn't open, print an error:
Serial.println("error opening file.txt");
}
}
void loop() {
// nothing happens after setup
}
If it is a Uno or similar remeber there is only 2K of ram and that the SD buffer takes up 512K, so trying to have an array taking 1800 bytes is not going to work.
Post the output of the compiler (there is a button on the RHS of the IDE to copy it all.
Sketch uses 10776 bytes (33%) of program storage space. Maximum is 32256 bytes.
Global variables use 978 bytes (47%) of dynamic memory, leaving 1070 bytes for local variables. Maximum is 2048 bytes.
This is independent of the size of the array that I initialize. Or if I change it to a 16bit integer, the compiler will give me these memory usage numbers. But for my ledstrip purposes value between 0 and 255 is enough, so I could work with the uint8_t values, which should be 1 byte each right, so should I theoretically be able to load an array of 1070 units?
You have 1070ytes free of which the sd file buffer will take 512 once initialised so leaving about 558 but in practice less since there are a few more variables needed by the SD instantiation. You might get away with a 500 element array of single bytes ut even that maybe tight.
This looks like you are just starting a new project, you may be better off coming up with an alternative way of processing the file which does not read it all into memory first, or chaning processor to a Mega (8K irc) or even to an esp8266 or ESP32 etc,
If you explain what you intend to do, it is possible we can point you in the right direction.
Okay I see. I thought those would be accounted for already in the message from the compiler.
I have a 300-led addressable ledstrip. I want to project an 'image' on there basically line for line in a loop, so each iteration I have 900 values. In my first attempt I tried storing them in a large array in PROGMEM, but then I could only store about 16 lines, which was about 10% of 1 image. Ideally I would like to have different modes for different functions. (I'm using a Gameduino touchscreen to control these modes). I originally had the idea that I could use the micro SD to store more of the PROGMEM values, but didn't work obviously.
So my plan now was to read in 1 line (900 values) at a time from the micro sd, and display those, and then read in the next line for the next iteration. I'm not sure yet if that even will be fast enough to make a smooth animation, but I thought it was worth a shot.
You might be able to read in say 300 values, process then the next 300 values and so forth. Reading an SD card file sequentially can be quite quick the sd library calls the SDfat library I believe which will buffer 512 bytes of data at a time, so reading 512 bytes then the remainder might be the fastest option. I have no idea how long that will take though, but it will be very dependant on your particular SD card. It might be worth you carrying out a few tests on data on your sd card and getting timings from before and after you read the data to give you the number of milliseconds it takes to read the first 512 bytes, then the remainding 388. Yo might also want (just out of curiosity) perform timings on how long it takes to your LED strip. Post the results up here and there may be some ideas on how to best improve performance. Since you are looking at multiple lines of 900 bytes most should be covered in just 2 reads, if the ones that take 3 are noticeably slower you might even consider padding everything out to 1024 bytes so that every line requires just 2 reads.
If you look in the SDfat library examples, there should be some that can give you an idea of how fast an SD card can read these things, when reading sequentially I suspect it will be significantly faster than you require.
You don't need to read the full line. In theory you can read the three values, convert and put it in the neopixel array, read the next three, convert and put them in the neopixel array and so on. Once you have read the info for each pixel, execute the show function of the library that you use.
Not sure how fast it is. You can also use an SPI or I2C EEPROM or FRAM module for extra storage.
Note
Rather use a more powerful micro as already indicated. 900 bytes for just the strip is a lot.
The 900 byte buffer for the LED strip along with the 512 byte buffer for the SD card will use 1412 bytes of RAM, might work but you will need to be careful with the rest of the sketch. It will help if you store the data on the SD card as binary data instead of text, that would eliminate the need for parseInt() and make data transfer a bit faster.
How many 900-byte "frames" does the animation require?
My Mega arrived a couple days ago, so that's a relief already
I can now without trouble read in the 900 values. I tried to time how long it takes to read them in,
which was about 400ms for the 900 values. It takes my script about 5ms to run every iteration/frame and then I originally had a delay of 100ms in between frames. So if I would implement it like this, it would slow down the 'animation'.
david_2018:
It will help if you store the data on the SD card as binary data instead of text, that would eliminate the need for parseInt() and make data transfer a bit faster.
How would I then read/store them? which function should I look into a bit more?
david_2018:
How many 900-byte "frames" does the animation require?
For the test image that I made of the Starry Night, that one had 169 frames. but I would like to be able to have different images as well.
Lastly, for now, is there a way to tell the sd-read function to start at a certain point in the text file?
or would I need to make a new text file for each frame.
kind regards
If all the records in the file are the same number of characters then you can calculate the position in the file you want to read from and use file.seek(position) to position the cursor at that position so the next read gives you want you want.
You can use file.write(buf,len) to write binary data and file.read(buf, len) to wriate and read binary data.
Note that keeping the file open if you are going to read multiple frames will be significantly faster than closing and opening the file. Not clear if your time to read in 900 values includes the file opening or just the reading of 900 values or average time to read 900 values.
I would think the fastest way would be to store the images as binary data, then read all 900 bytes directly into the led buffer using .read() . Locating a specific frame of the image in the file would then be simple, since each frame is 900 bytes. Reading in the numbers as ascii text is going to be much slower, since each number can be up to three characters, plus whatever character is separating the numbers.
as it is now, they are not the same number of characters, as some values are '11' and some '255', but that could easily be fixed by adding 0's. But I understand that using bytes is probably quicker anyway. Lets say I would have 300 rgb pixel values of (255, 200, 100) as an example.
Can I then make a text file (I'm using a python script for that, nothing with Arduino itself) that looks like this:
11111111 11001000 01100100 ... ... ...
for the first frame I could then use file.read(buf, len) where I say that the length of the buffer is 900?
and then for all the other frames, I first use the file.seek(900*frame) and then the file.read(buf, len)?
would it work like this, with the whitespaces as seperators?
You are still thinking in terms of text. A byte can contain a value between 0 and 255. When you use a string each character is a byte and can contain a value between 0 and 255.
You need to write a binary file with no separators, just one byte after another. If you look at the file contents it will most likely look like jibberish. There is no conversion from a byte to characters when written as binary and when read each byte is read and moved directly into your buffer - again no translation. It is the translation from binary to human readable text (or vice versa) that slows the process down.
