Parsing data from file

I'm reading a file in from an SD card. The data is stored as follows:


REPS 172
ELONG 1411

Each line of values has 48 of them on a single line. The number of lines of values varies with each file. All files end with the same four lines at the bottom, EMPTY_LINE, REPS, PAUSE, and ELONG.

I need to somehow read all of that info in, set the necessary variables for REPS, PAUSE, and ELONG, and have the values shoved into PROGMEM like so:

prog_uchar image[] PROGMEM = {0x52,0x52,0x52,0x52,...,0x7E,0x7E,

As far as reading the file in, I can do that already. I can read the file line by line and echo it back out on the Serial monitor. But I don't know how to proceed from here as far as parsing the information and shoving it into PROGMEM.

Suggestions anyone?

PROGMEM data can only be set at compile time, you are limited to a ram based array.

PROGMEM data can only be set at compile time, you are limited to a ram based array.

Fair enough. Let's go with that. How would I go about parsing that data into an array?

As you read the data in, detect the 0x, then using a function similar to this ( convert hex nibble to value ) you can compile a value:

char GetValue( char c ){

  if( c >='a' || c <= 'f' )
    c -= 'a' + 10;
  else if( c >='A' || c <= 'F' )
    c -= 'A' + 10;
  else if( c >='0' || c <= '9' )
    c -= '0';
  return c;

Then somewhere in you incoming data code.

//Detect 0x

//Get next two chars into first_char, second_char.

unsigned char result = ( GetValue( first_char ) << 4 ) | GetValue( second_char );

//Store result in array.

You need to address the RAM space issue - your Arduino has very limited RAM and if you are storing an unknown number of 48-byte chunks of data you could easily exceed it. After writing your sketch you will need to measure the amount of free RAM, work out the maximum amount of data you can store in the remaining space, and decide how you're going to handle the case where you receive more than you can store.

If you are dealing with a lot of data and not doing anything time-critical, it would probably be easier and safer to load the data records from the file as you need them rather than try to store the whole lot in RAM.

Wouldn't it be easier to use strtok() from the standard library using the comma and newline to parse the data? More info at:

While you probably already know this, I forgot to mention that strtok() works on strings so you would need to read in the line and replace the newline character at the end of the line with the string termination character (i.e., null, or '\0').