Go Down

Topic: SD read and store (Read 2 times) previous topic - next topic

mrquacks

Hello,

I'm pretty new to Arduino code, though I've been doing Java for a while... makes this a little confusing. What I'm trying to do is read single lines from an SD and then split the data based on various delimiters and store them. For each line, there's a keyword and several associated words, so I want to store the key in a HashMap and have the value be a Queue of the other words.

I found premade HashMap and QueueList libraries (which have worked for me thus far) but I've run into a problem.  I can successfully read the data and store the key in the table and I can store the other words into a QueueList, but I can't seem to store the QueueList with the key.

For example, in the following code (***note database is my HashMap variable name, and split is function I have for splitting strings), the print statements at the bottom (resp.pop()) all print out the correct thing. I've also checked the HashMap and they key is successfully stored, but when I try to print something like database.valueAt(key).pop(), it prints a blank line.
Code: [Select]
void setup()
{
 Serial.begin(9600);
 Serial.println("Start");
 
 SD.begin(4);
 File file = SD.open("brain.txt");
 
 if (SD.exists("brain.txt")) {
   Serial.println("Exists.");
 } else {
   Serial.println("boo.");
 }
 
 char buff[500];
 memset(buff, 0, sizeof(buff));
 int idx = 0;
 while (file.available())
 {
   char c = file.read();
   if (c != '\n' && c != '\r')
   {
     buff[idx] = c;
     idx++;
   }
   else {
     QueueList<String> resp;
     int rNo = 1;
     for (int i = 0; i < strlen(split(buff, ":", 2)); i++) {
       if (split(buff, ":", 2)[i] == '/') {
         rNo++;
       }
     }
     for (int i = 1; i <= rNo; i++) {
       resp.push(split(split(buff, ":", 2), "/", i));
     }
     database[split(buff, ":", 1)] = resp;
     Serial.println(resp.pop());  
     Serial.println(resp.pop());
     Serial.println(resp.pop());  
     Serial.println(resp.pop());
     memset(buff, 0, sizeof(buff));
     idx = 0;
   }
 }
}


I'm afraid the issue is something with passing values or whatnot, but I'm not sure.

Thanks for your help!

WizenedEE

Well, you have 2048 bytes of ram. The SD card library uses 512 bytes, and your buf arrays uses another 500. I'm guessing the other two libraries you're using also use up quite a bit. So, I think the problem is here:

Quote
I've been doing Java for a while


You're going to need to give up some stuff to save memory when working on microcontrollers.

Eg:
You're going to need to get rid of buf and read one line at a time and find the delimiters in the stream.
Both resp and database seem to be storing huge amounts of things on the Heap little by little. This makes memory fragmentation run rampant. Get rid of them if at all possible.

PaulS

Posting code snippets only makes it very difficult to help you. How is database defined? what does the split() function do?

What does a sample line from your file look like? How many lines are there?

wildbill

Given your apparent memory needs, you might find it easier to use a Mega (assuming you're not already) to get more RAM. I assume you're concentrating on the config load piece for now; what will you be doing once you have this working (i.e. what is your project supposed to do) and is there any way to store that config data in a less heap intensive way?

mrquacks

Hey thanks for the replies.

I'm working on a Mega and have already bought a QuadRAM attachment which will add 512 kilobytes of RAM. This was just a testing sketch to see if I could properly read in data... the split function is the only thing I didn't include and it looks as follows:

Code: [Select]
char* split (char* str, char *delim, int index)
{
  char *act, *sub, *ptr;
  static char copy[100];
  int i;

  // Since strtok consumes the first arg, make a copy
  strcpy(copy, str);

  for (i = 1, act = copy; i <= index; i++, act = NULL)
  {
    sub = strtok_r(act, delim, &ptr);
    if (sub == NULL) break;
  }
  return sub;
}


I found it somewhere else on the arduino forum. As for one line of the txt file.. thats formatted like:

Code: [Select]
keyword:word1/word2/word3

The reading in is fine and the splitting is fine (from what I've tested). The storing is even fine--Just not when I try to call a specific QueueList out of the HashMap. The QueueList is found here: http://arduino.cc/playground/Code/QueueList and the HashMap was taken from http://wiring.uniandes.edu.co/source/trunk/wiring/firmware/libraries/HashMap/

If you think its a problem with RAM, the part should arrive today or tomorrow and I can continue debugging.  I was just wondering if you saw anything obviously wrong in the code. Ultimately, this will be used to store responses to certain actions (and there are a lot of them) for a stand alone Arduino board, so I thought this would be the easiest way to do that since it won't be connected to a computer.  Learning the limitations of a microcontroller is definitely something I should do, but really this project is more about a proof of concept in a limited amount of time than being extremely efficient. Thanks!

Go Up