SD Card read/write help

Projects Programming
1

Hi
If i had a test.txt file containing:

{191,64,254,128,4,606,378,321},
{191,128,254,128,4,607,378,321},
{191,1,254,128,4,608,378,321},
{127,2,254,128,4,609,378,321},
{127,4,254,128,4,610,378,321},
{127,8,254,128,4,611,378,321},
{127,16,254,128,4,612,378,321},

on an SD card properly connected to a Mega, how can I read it into an array named data?

So the array would look like:

int data[7][8] =
{
{191,64,254,128,4,606,378,321},
{191,128,254,128,4,607,378,321},
{191,1,254,128,4,608,378,321},
{127,2,254,128,4,609,378,321},
{127,4,254,128,4,610,378,321},
{127,8,254,128,4,611,378,321},
{127,16,254,128,4,612,378,321},
};

2

By parsing it in step by step. You can use functions like strrok() and strtol() for that.

3

Please give an example I,m getting the following which has a 1 added to the end of each character:
{11
91
11
,1
11
21
81
,1
21
51
41
,1
11
21
81
,1
41
,1
61
01
71
,1
31
91
61
,1
31
21
21
}1

4

You haven't posted your code. How can we guess why you're getting that result?

Also, have you investigated the functions I mentioned yet?

5

myFile = SD.open("test.txt");
if (myFile) {
Serial.println("test.txt:");
Serial.write(myFile.read());
// read from the file until there's nothing else in it:
while (myFile.available()) {

int data = Serial.write(myFile.read());
Serial.println(data, DEC);
}
// close the file:
myFile.close();
}

6

How to parse a data file

7

That's not your code. Please post full code and please use code tags when posting code; see the 'how to use this forum' sticky to find out more.

And the one more than likely comes from your last serial print.

8

Obviously I modified the example code to meet my purpose as in learning through practice by sample/trial error.

/*
  SD card read/write

 This example shows how to read and write data to and from an SD card file
 The circuit:
 * SD card attached to SPI bus as follows:
 ** MOSI - pin 11
 ** MISO - pin 12
 ** CLK - pin 13
 ** CS - pin 4

 created   Nov 2010
 by David A. Mellis
 modified 9 Apr 2012
 by Tom Igoe

 This example code is in the public domain.

 */

#include <SPI.h>
#include <SD.h>

File myFile;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }


  Serial.print("Initializing SD card...");

  if (!SD.begin(53)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");

 

  // re-open the file for reading:
  myFile = SD.open("test.txt");
  if (myFile) {
    Serial.println("test.txt:");
Serial.write(myFile.read());
    // read from the file until there's nothing else in it:
    while (myFile.available()) {
      
  int   data= Serial.write(myFile.read());
  Serial.println(data, DEC);
    }
    // close the file:
    myFile.close();
  } else {
    // if the file didn't open, print an error:
    Serial.print("error opening test.txt");
  }
}

void loop() {
  // nothing happens after setup
}
9

OK problem solved, I have successfully modified the code here :

http://forum.arduino.cc/index.php?topic=340849.msg2349945
post 2 by fat16lib

except I,m getting:

191 64 254 128
191 128 254 128
191 1 254 128
127 2 254 128
127 4 254 128

but I need to get the data in this format so I could use it in an array of type below:
const uint16_t data[][4] PROGMEM =
{
{191,64,254,128},
{191,128,254,128},
{191,1,254,128},
{127,2,254,128},
}

// Example to read a two dimensional array.
//
#include <SPI.h>
#include <SD.h>
#define CS_PIN 53

// 5 X 4 array
#define ROW_DIM 5
#define COL_DIM 4

File file;

/*
 * Read a file one field at a time.
 *
 * file - File to read.
 *
 * str - Character array for the field.
 *
 * size - Size of str array.
 *
 * delim - String containing field delimiters.
 *
 * return - length of field including terminating delimiter.
 *
 * Note, the last character of str will not be a delimiter if
 * a read error occurs, the field is too long, or the file
 * does not end with a delimiter.  Consider this an error
 * if not at end-of-file.
 *
 */
size_t readField(File* file, char* str, size_t size, char* delim) {
  char ch;
  size_t n = 0;
  while ((n + 1) < size && file->read(&ch, 1) == 1) {
    // Delete CR.
    if (ch == '\r') {
      continue;
    }
    str[n++] = ch;
    if (strchr(delim, ch)) {
        break;
    }
  }
  str[n] = '\0';
  return n;
}
//------------------------------------------------------------------------------
#define errorHalt(msg) {Serial.println(F(msg)); while(1);}
//------------------------------------------------------------------------------
void setup() {
  Serial.begin(9600);

  // Initialize the SD.
  if (!SD.begin(CS_PIN)) {
    errorHalt("begin failed");
  }
  // Create or open the file.
  file = SD.open("READNUM.TXT", FILE_WRITE);
  if (!file) {
    errorHalt("open failed");
  }
  // Rewind file so test data is not appended.
  file.seek(0);

  // Write test data.
  file.print(F(
//    "11,12,13,14\r\n"
//    "21,22,23,24\r\n"
//    "31,32,33,34\r\n"
//    "41,42,43,44\r\n"
//    "51,52,53,54"     // Allow missing endl at eof.
"191,64,254,128\r\n"
"191,128,254,128\r\n"
"191,1,254,128\r\n"
"127,2,254,128\r\n"
"127,4,254,128"
    ));

  // Rewind the file for read.
  file.seek(0);

  // Array for data.
  int array[ROW_DIM][COL_DIM];
  int i = 0;        // First array index.
  int j = 0;        // Second array index
  size_t n;         // Length of returned field with delimiter.
  char str[20];     // Must hold longest field with delimiter and zero byte.
  char *ptr;        // Test for valid field.
  char delim = 0;   // Delimiter from previous line. Start with no delimiter.

  // Read the file and store the data.
  while (true) {
    n = readField(&file, str, sizeof(str), ",\n");

    // Read error or at EOF.
    if (n == 0) {
      break;
    }
    // Advance indices based on previous delimiter.
    if (delim == '\n') {
      // previous delimiter was endl so start a new row.
      if (++i >= ROW_DIM) {
        errorHalt("too many lines");
      }
      if (j != (COL_DIM - 1)) {
        errorHalt("missing field");
      }
      j = 0;
    } else if (delim == ',') {
      // previous delimiter was comma so advance column.
      if (++j >= COL_DIM) {
        errorHalt("too many fields");
      }
    }
    array[i][j] = strtol(str, &ptr, 10);
    if (ptr == str) {
      errorHalt("bad number");
    }
    // Skip any blanks after number.
    while (*ptr == ' ') {
      ptr++;
    }
    // Save delimiter.
    delim = *ptr;

    if (delim != ',' && delim != '\n' && delim != 0) {
      errorHalt("extra data in field");
    }
    if (delim == 0 && file.available() != 0) {
      errorHalt("read error or long line");
    }
  }
  // Check that entire array was read.
  if (j != (COL_DIM - 1) || i != (ROW_DIM - 1)) {
    errorHalt("missing data");
  }
  // Print the array.
  for (i = 0; i < ROW_DIM; i++) {
    for (j = 0; j < COL_DIM; j++) {
      if (j) {
        Serial.print(' ');
      }
      Serial.print(array[i][j]);
    }
    Serial.println();
  }
  Serial.println("Done");
  file.close();
}
//------------------------------------------------------------------------------
void loop() {
}
10

You can't write to PROGMEM at run time. So, you will not be able to populate that array with data from the SD card.

11

Thank you for the heads up, what is then the alternative?
This is for like a textile machine that the SD card is to hold the whole cartoon of say 2000 lines and each line is 800 fields hence the need for feeding 1 line at a time in an array of 800 rows by 8 fields and each field max 4 digits.
thanks
p.s.
the cartoon is like an image of 800 by 2000 pixels and each pixel of it is defined by an array row of 8 fields(hence 800 rows by 8 fields array) and I need to read the pixels from SD card 800 pixels at a time.

12

"You can't write to PROGMEM at run time."
Sure you can. One just needs a bootloader change:
https://forum.arduino.cc/index.php?topic=332191.0

Might need a processor like 1284P with 16K SRAM to hold it all.

13

I need to read the pixels from SD card 800 pixels at a time.

Lets talk data types. A "field" is not a data type. Why does the data need to be persisted in memory? Read a record, send the data to the machine, and forget it.

What does ONE record actually look like?

14

each pixel is defined like below:
{191,2,254,1,2,1,201,322},
the actual array for each line/row of the image needs to hold 800 lines of the above. The array could be broken down into smaller chunks but obviously a little more messy and more reads from the SD card.
edited
The 4 1st fields above are binary numbers and the last 4 decimals 0 to 9999.

15

the actual array for each line/row of the image needs to hold 800 lines of the above.

But why do you need all 800 sets of data in memory at the same time?

16

during the textile ops. the operator navigates back and forth across the 800 pixels. It can be piecemeal-ed but much preferred not to.

p.s. edited
here's what my mega shows using progmem and this is just for a 576x8 array:
Sketch uses 18,426 bytes (7%) of program storage space. Maximum is 253,952 bytes.
Global variables use 752 bytes (9%) of dynamic memory

17

You are reading the numbers successfully. All you have to do is read them into an unpopulated array of the correct size. You can use a nested loop to do that.
Your declaration is the same but with no initialization:

int  data[7][8];

Then you fill it with your data using a nested loop, e.g. i=0 to 6 and j=0 to 7, while you read the file.
You would populate an element like this:

data[i][j] = theNumberThatIreadFromTheFile;

Please go away and think about this before you reflexively ask how to do it.

18

Thanks a lot, I will start studying this right away.

19

Cyquant:
Thanks a lot, I will start studying this right away.

Also if you face memory constraints, consider whether you can use portions of the file at a time, instead of reading the entire thing into RAM. If the "textile machine" only weaves one line at a time, then that's all you need to read until it's complete.

20

The machine needs an array of 800 x 8 to do 1 complete line of it's work before going to the next line which would be fair to do an SD read then but since without progmem the above array doesn't fit into the Mega I will have to feed the machine like say an array of 200 x 8 at a time hence rather than just 1 SD read per line it will have to do several reads per line of weaving which is not as clean cut as just 1 read and also may slow down the process because of several SD accesses rather than just 1.