how to go about using an sd card to store a list to be read from

Using Arduino Project Guidance
#1

For my project I am going to need to store a large list of words (around 1000) that will then be randomly chosen by the arduino to be displayed. The list was going to be stored on an SD card and I was wondering the best way to do that.

I am curious if there would be possible for me to create an excel spreadsheet on my computer with all the words, save it to the card, then if there would be a way for me to read that spreadsheet using the arduino. My goal would want to generate a random number and use that number to correspond to the row that would have the word to be displayed.

Can spreadsheets be used like this? Is there some other way for me to store a list of words with a corresponding number for look up?

Any advice is helpful! Thanks in advance!

#2

I think this is very unlikely. A spreadsheet is like a webpage, in that you see a visual image on the screen, but that is actually an active GUI of a program, and in the case of a spreadsheet, it's a very complex program that you couldn't possibly stuff into an Arduino. SD card libraries all seem to allow you to use the SD card storage organized into files, so I suggest if you want access to a list of words, you store them as a flat text file in the arduino, with some indexing attached to allow you to seek a word from anywhere in the list. But on an Arduino, with a long list, this won't be very speedy.

#3

Just a hobby coder, but I think you should be able to store that amount of words into Arduino’s flash memory.
Like this:
const char * const wordList[1000] PROGMEM = {“one”, “two”, “three”, “four”,etc…};
This page could help.
http://www.gammon.com.au/progmem
Leo…

Edit: Just for fun I tried to store 10,000 words into an Uno.
Not a problem. This used only 68% of the memory.

const unsigned int numberOfwords = 10000;
const char * const wordList[numberOfwords] PROGMEM = {
  "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen", "twenty", "twentyone", "twentytwo", "twentythree", "twentyfour", "twentyfive",
  // 398 more of those 25-word lines
  // copy/paste if you want to try it
  //
  "one", "two", "three", "four", "five", "six", "seven", "eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen", "twenty", "twentyone", "twentytwo", "twentythree", "twentyfour", "10,000",
};
unsigned int count = 0;

void setup() {
  Serial.begin(115200);
}

void loop() {
  for (int i = 0; i < numberOfwords; i++) {
    char * Word = (char *) pgm_read_word (&wordList [i]);
    Serial.print (Word);
    Serial.println(" ");
    delay(5);
    count += strlen (Word) + 1;
  }
  delay(10000);
}
#4

Right now, as a way to test the other functions of my program, I use a switch statement to generate(store? Im not sure what the proper terminology is here) all of my words then get a random number to determine which case to use. I feel like that would be inefficient to expand from the 20 words I have as a proof of concept to the 1000 I want for functionality. My inclination was to just generate a random number and use that number to find the word in the sd card.
Is there a command or process that would read from an SD card and then return the string of the appropriate indexing?

Im not sure if this would impact using program memory, but I print these words to an LCD.
Therefore I use an LCD.print(); command. Would it be possible to ignore the SD card and use program memory space if i used an LCD.print(F()); command instead? my current program uses 6000 bytes out of the 32000 available.

#5

You could use a text editor and put one word on each line, in other words, delimited by a newline character. Then to index a word, you open the file and read and discard n-1 lines until you come to line n that you are looking for. It will contain the word you are indexing. Close the file and begin again if you want another one. It will be somewhat slow.

#6

If this is all What your arduino needs to do and the 1000 words don't need to be changed often (because need of reflashing the pgm) then I second the PROGMEM approach. No need for attaching a SD card and mess around with slow data access. PROGMEM is really only worth it if you have a larger block of data that needs to be stored, which is usually easiest in an array and this is your case.

An UNO has 32kbyte of flash so with words of average length of say 10 characters, you use less than a third of the space. The good thing about this is that your coding then is pretty simplified - see the link above on accessing the array members.

If you want the flexibility of uploading data in a file on a removable external storage such as a SD card, then I would recommend having a very strict file structure with a word per line and each word being allocated the same number of characters - fill in with a special character your words won't use - may be a '@' sign for example - before or after your word and add a new line at the end of the padded word. You can use excel to have in column 1 your list of word, create a function in column 2 using excel for building the padded word (simple functions on strings, possibly put all lower case or uppercase as well, whatever suits you), then select all words from column 2, copy and paste in a text editor and save the file as UTF8 format (8 bits per character). Don't use a RTF, Word, excel or any other file format to save the list, it will be way more difficult to extract your words otherwise because of the Unicode and extra formatting characters or instructions in the file format.

The reason for having fixed length strings padded with '@' is that

  • you have limited RAM in your arduino, 1000 words won't fit in memory
  • thus you need to find the right line in your file but if you want fast access to a given word, you don't want to read line by line incrementing a counter until you get to the right line number. That will be fast for word #1, that will be 1000 times slower to find the one at position 1000. Not good.
  • smallest SD cards you get those days have at least 1 or 2 Gig of data so you could store the full dictionary with padding without problems in there, so use that space, you paid for it anyway :slight_smile:
  • AND fixed string size makes math simple to calculate where to point to in your file to access a random word - like number #287 - without reading and parsing each line of the file. Just do 287 x nb_char_in_your_strings (including the carriage return and possibly line feed depends how your text editor saves new lines) and you have the index (starting at 0) where to start parsing the file on SD Card. Open the file, jump at that index, read a line in a buffer or better byte by byte ignoring the padding chars until you find the end of the line (don't save those, put a zero in the buffer instead). buffer is of the same size as your nb_char_in_your_strings. Don't forget to close the file.

if you did not do it at reading time ( you should), get rid of the padding and end of line characters - if you padded in front of the string, then a simple for loop to find the first '@' sign will give you the index of where your string really starts (if first char is not '@' then it means the word did not need padding and you already have the whole word). Ensure you deal with end of line - probably want to get rid of those in your string (the '\n' and /or '\r') and replace those with zero at the end of your buffer to get a properly formatted C string, that will make it easy to print using strandard library print commands.

Hope this helps.

#7