Working with SD Card - CSV Files

Using Arduino Storage
1

I only recently started working with C and Arduino, and in this environment I have to be mindful of the limited memory resources available. Code has to be precise and efficient.

I wrote a basic function/method to read time zone data from a micro SD card CSV file:

void GetTimeZones(char* fileName)
{
String s1 = "";
String s2 = "";
if (SD.exists(fileName) == false)
{
Serial.println("File Not Found!");
return;
}
File myFile = SD.open(fileName, FILE_READ);
if (!myFile)
{
Serial.println("Open File Failed!");
return;
}
myFile.seek(0);
while (myFile.available())
{
s1 = myFile.readStringUntil(',');
s2 = myFile.readStringUntil('\n');
Serial.print(s1);
Serial.print('\t');
Serial.println(s2);
}
myFile.close();
}

This function works as expected, however my question to the forum is this:

In many cases I need to present the user with a selection list, using a character display. To do this I am using a rotary encoder for the user input to cycle through the list. Pressing the button makes the selection.

In the above example I'm using time zones loaded from a file, in another case it's a list of WiFi networks to chose from.

If the list is small enough I can create an array an use an index into that array for current display item and selection. But in the above example the data is too large for available memory, so I need to take a different approach.

I don't think reading the file to get a single item on each interrupt of the rotary encoder will be fast enough, especially if the encoder is turned quickly.

Can anyone advise what my best approach might be?

2

Code has to be precise and efficient.

Using the String class is not efficient.

I don't think reading the file to get a single item on each interrupt of the rotary encoder will be fast enough, especially if the encoder is turned quickly.

No, it probably won't be.

Can anyone advise what my best approach might be?

Shorter values. An Arduino with more memory.

3

Appreciate your feedback, but not much help to me. I was really after a methodology - an approach to use to achieve the job of user selection from a long list of data using an encoder.

As I know the length of data I'm reading from the file, I can use static allocation and create a simple array of chars: char blah[] = "Single";

I'm using an Arduino Mega 2560 which has 256KB of flash memory. Minus the 8KB used by the Bootloader, that leaves me with 248KB for code, which is enough for what I'm doing.

But variables instantiated by the code are stored in SRAM, of which there is only 8KB! Why so little? Surely its not a space issue.

In some instances the data in my file is around 20K. I do have the option of caching small chunks of the data at a time, but that gets more complicated and calls for more code as the user can scroll the encoder forward and backward and at varying acceleration.

As I don't need to modify the strings or data while my app (sketch) is running, I think my best bet will be to store the data in flash (program) memory instead of SRAM.

4

Why so little? Surely its not a space issue.

For the purposes the chip was designed for, like running your washing machine, 8K is plenty.

As I don't need to modify the strings or data while my app (sketch) is running, I think my best bet will be to store the data in flash (program) memory instead of SRAM.

In which case the issue of how to rapidly read from a SD card becomes immaterial.

5

OK thanks. I managed to get the rotary encoder and list selection working happily.