Pages: [1] 2   Go Down
Author Topic: reading text from an sd card  (Read 1608 times)
0 Members and 1 Guest are viewing this topic.
0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm playing with the sd card read/write tutorial.  The example sketch works fine.

When I create a text file in notepad, copy it to an sd card and open it using the arduino, the formatting is different.  Ie -

Original text - this is a test 1 2 34
Text printed in serial monitor - tisisa es 12 4

I've tried asni, utf8 etc

any ideas?
Logged

Switzerland
Offline Offline
Faraday Member
**
Karma: 108
Posts: 5144
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Show us your code.
Logged

Dallas, Texas
Offline Offline
God Member
*****
Karma: 30
Posts: 887
Old, decrepit curmugeon
View Profile
WWW
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm playing with the sd card read/write tutorial.  The example sketch works fine.

When I create a text file in notepad, copy it to an sd card and open it using the arduino, the formatting is different.  Ie -

Original text - this is a test 1 2 34
Text printed in serial monitor - tisisa es 12 4

I've tried asni, utf8 etc

any ideas?

Original text - this is a test 1 2 34
Serial text    -  t  is is a  es  1 2    4

As previously mentioned code would help; however, the serial text is a subset of the original which would indicate data being dropped, either in the serial transmission to the computer or possibly in your code reading the SD card.
Logged

New true random number library available at: http://code.google.com/p/avr-hardware-random-number-generation/

Current version 1.0.1

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Ha, I just rewrote the code and it works. Not sure what it was as the code is mostly just the example sketch.

One question, this code is printing to serial using Serial.write, not Serial.print.  As I understand it this is because the data is in binary form. Given this, how do I assign the text (ascii) contents of this file to a string?


Here is the code -

Code:
#include <SD.h>

File myFile;

void setup()
{
  Serial.begin(9600);
  Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output
  // or the SD library functions will not work.
   pinMode(53, OUTPUT);
   
  if (!SD.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
 
 
  // re-open the file for reading:
  myFile = SD.open("test.txt");
  if (myFile) {
    Serial.println("test.txt:");
   
    // read from the file until there's nothing else in it:
    while (myFile.available()) {
    Serial.write(myFile.read());
    }
    // close the file:
    myFile.close();
  } else {
  // if the file didn't open, print an error:
    Serial.println("error opening test.txt");
  }
}

void loop()
{
// nothing happens after setup
}

Logged

Germany
Offline Offline
Faraday Member
**
Karma: 56
Posts: 2983
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The corresponding method to Serial.read() is Serial.write().

Additionally, write() can also send null terminated strings, if you provide a char* parameter.
Serial.print()  and println() are rather convenience methods to format various data types into readable text.

To read longer texts from a file into variables you have to take care where to store it ( and how long ), as Arduino RAM is very limited.

The sample code does not store it at all, but simply uses every read character just as output parameter to Serial.write().
This way, it can handle files of any length.

For any other use, the solution depends on what you need.
Logged

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I want to read in the text  "46,110,173,237;"

and store it as a String "46,110,173,237;"
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 12
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

See my question: "Problem with DmxSimple.h". My code does exactly what you want and then even parses the lines of text. Of course it seems to be un-usable when including DmxSimple.h but that is another topic :-)

Regards

Simon
Logged

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I'm trying to make the same thing smiley
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 12
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Well a popular thing to do I guess :-). Thanks to pylon my code now works. I have placed a lot of the debug messages in comments aqnd now the codes works fine even with the include DmxSimple.h You can clean it up as this was my "test and debug" version but it does what it should. You'll still have top add the proper calls to DmxSimple but at least the lines are parsed correctly and are available in an array.

Here is the "new" version of my code:
Code:
//SD CSV File Read

#include <SD.h>
#include <String.h>
//#include <DMXSerial.h>
#include <DmxSimple.h>

//Set by default for the SD Card Library
//MOSI = Pin 11
//MISO = Pin 12
//SCLK = PIN 13
//We always need to set the CS Pin
int CS_Pin = 10;
//int PowerPin = 8;

// Declare DMXValues Array and DMXValueIndex Counter. Keep Unsigned Integers below 65535
unsigned int DMXValues[15] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
int DMXValueIndex = 0;
int IterationCounter = 0;
int AvailableCounter = 0;

// Declare CSVStringBuffer Input Character Buffer, SimpleToken "container" and IntermediateBuffer Temporary Location for Read char from File
char CSVStringBuffer[80];
char *SimpleToken = NULL;
char IntermediateBuffer = NULL;

void setup()
{
  Serial.begin(9600);
  Serial.println("Initializing Card");
  //CS Pin is an output
  pinMode(CS_Pin, OUTPUT);
 
  //Card will Draw Power from Pin 8, so set it high
  //Strange Arduino Idea... Arrrrggghhh them Software Designer they're so naughty!
  //This May not survive in Final Design
  //pinMode(PowerPin, OUTPUT); 
  //digitalWrite(PowerPin, HIGH);
 
  if (!SD.begin(CS_Pin))
  {
      Serial.println("Card Failure");
      return;
  }
  Serial.println("Card Ready");
 
  // Read DMX Sequence information (CSVTest1.csv)
  File CSVCommandFile = SD.open("CSVTest1.csv");
  if (CSVCommandFile)
  {
    Serial.println("Reading DMXSequence File");
   
    AvailableCounter = CSVCommandFile.available();
    Serial.print("Available Character(s): ");
    Serial.println(AvailableCounter);
   
   // As long as there are characters and that the Index is less than 80 characters read the file
   while(CSVCommandFile.available() && (IterationCounter <= 79))
   {
     IntermediateBuffer = CSVCommandFile.read();
      // Look for the first exception, 0x0D or carriage return. May be absent in a Mac generated CSV file
     if (IntermediateBuffer == 0x0D)
     {
       continue; // If found, re-iterate the while loop
     }
     // Look for the second exception, 0x0A or line feed
     else if (IntermediateBuffer == 0x0A)
     {
       CSVStringBuffer[IterationCounter] = NULL;
       
       
       // Actual Parsing of CSVStringBuffer to find Tokens
       // This is where the magic happens :-)
       SimpleToken = strtok(CSVStringBuffer, ",");
       while(SimpleToken)
       {
         // Do something with this token
         DMXValues[DMXValueIndex] = atoi(SimpleToken);
         
//         Serial.print(DMXValues[DMXValueIndex]);
//         Serial.print(" was read into Integer Array at position ");
//         Serial.println(DMXValueIndex);
         
         DMXValueIndex++;
         
         // Get the next token...
         SimpleToken = strtok(NULL, ",");
       }
       
       // Various progress messages
//       Serial.println("Done Acquiring Character from 1 Line in CSVTest1.csv");
       
       Serial.print("Integer Array is: ");
       for(int j = 0; j <= (DMXValueIndex - 1); j++)
       {
         Serial.print(DMXValues[j]);
         Serial.print(" ");
       }
       Serial.println(" ");
       
       // Reset Index for Values Array before exiting and re-looping for the next line in CSVCommandFile
       DMXValueIndex = 0;

       // Reset Index for CharBuffer before exiting and re-looping for the next line in CSVCommandFile
       IterationCounter = 0;
     }
     
     // This section is where the CSVStringBuffer is filled with the characters from the file
     else
     {
       CSVStringBuffer[IterationCounter] = IntermediateBuffer;
//       Serial.println(CSVStringBuffer[IterationCounter]);
       //Serial.print("Iteration Counter = ");
       //Serial.println(IterationCounter);
       IterationCounter++;
     }
   }
  }
 
  // If upon exit of while loop i hasn't been reset there was a problem. Should test to determine the problem and print to log file
  if (IterationCounter != 0)
  {
    Serial.println("Aborted Character Acquisition");
    Serial.print("IterationCounter= ");
    Serial.println(IterationCounter);
//    Serial.print("Available Character(s): ");
//    Serial.println(AvailableCounter);
  }
  // Succes message
  else
  {
    Serial.print("Read total of ");
    Serial.print(AvailableCounter);
    Serial.println(" characters");
//    Serial.print("DMXValueIndex= ");
//    Serial.println(DMXValueIndex);
    Serial.println("Operation Succesful");
  }
}

void loop()
{
}

Logged

Germany
Offline Offline
Faraday Member
**
Karma: 56
Posts: 2983
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
I want to read in the text  "46,110,173,237;"
and store it as a String "46,110,173,237;"
Then it's slightly different to pshoule's approach ...

I guess your rules are:
- There are max 16 characters ( 4*3 digits + separators ). So define a char text[17]; as minimum for those characters and a trailing 0 end indicator.
- The last one is the only semicolon ( ';' ) in the file, and it's always there as an end indicator. (If you're at the end of the provided space, treat it as a wrong file)
- There are no line feed ( 0x0A ) or carriage return (0x0D) characters in the file before the semicolon. (If there are, treat it as a wrong file)

If you were interested in the numbers, there were other rules, of course. ...and pshoule's approach was worth having a closer look smiley-wink

You just want to read a character, store it in your text array at the correct location and check if all is still ok. If ok and no end indicator found yet, do it again with another character to the next location.
When done, add a trailing 0 after the last character you just stored ( the ';' ), so that Serial.println(text); works well.
« Last Edit: May 30, 2012, 02:05:31 pm by michael_x » Logged

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Ah haah, thanks. you've just helped me understand some crucial things!

how's this? -

Code:
int byteCount = 0;
      while (myFile.available())
      {
        testBuff[byteCount] = myFile.read();
        byteCount++;
      }
      testBuff[byteCount] = NULL;

Can I set the size of char testBuff[] dynamically, increasing by 1 every time I need to add a new byte, or do you have to just initialise the array to be larger than it can ever be?
« Last Edit: May 30, 2012, 02:40:14 pm by mrboni » Logged

Germany
Offline Offline
Faraday Member
**
Karma: 56
Posts: 2983
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
just initialise the array to be
...as large as it can ever be!
And check if that's true ( never write beyond the array size )

Quote
how's this?
Fine
Logged

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Great.

One more thing - As each comma separated value is describing one value of 0-255, is it possible to just send single bytes in sequence with a delimiter at the end?

If so, how would these appear in a text file? I'm using another piece of software to generate the text files and make it do anything. I'm just not sure what a single byte should 'look like'

This would reduce each file size by 75%, which would be great as I'm trying to send lots of values very quickly.

Thanks again
Logged

Germany
Offline Offline
Faraday Member
**
Karma: 56
Posts: 2983
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


Is it possible to just send single bytes in sequence with a delimiter at the end?

If so, how would these appear in a text file? I'm using another piece of software to generate the text files and make it do anything. I'm just not sure what a single byte should 'look like'
Sure, it's not a text file any more but some binary file, without any structure.
You cannot read it with notepad, but it would open and read in your sketch were the same.
Every byte you read would have a specific meaning.

This would reduce each file size by 75%, which would be great as I'm trying to send lots of values very quickly.

File size does not really matter, and won't affect speed too much, I think. It's not a Serial connection at 2400 baud.
What is "lots of values" ? You cannot store 2k bytes in an Arduino UNO anyway. If it's 200 bytes, it does not really matter if the file size is 200 or 1000 bytes.
I've got a very old SD card of 32 MB, which is horribly oversized for my Arduino project.

As a compromise: what about storing the data as hex strings. perhaps without comma delimiters but newline to structure it and make it readable?

"46,110,173,237;" would become
"2E6EADED"

Readable in a text editor, and easily convertible to binary in Arduino.
Logged

0
Offline Offline
Full Member
***
Karma: 0
Posts: 124
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi Michael.

I'm using a Mega2560 with an ethernet shield.

I've created text files, each one representing one 'frame' of 512 8bit values.  The plan is to read these load each frame into arduino ram, update these 512 values (which get sent as dmx), then load the next frame.

I need to read 512 8bit values at as close to 30 fps as possible.


Below is the test code I'm using to open then close 30 of these text files

Where do you think the bottleneck is?

Code:
#include <SD.h>

File myFile;

void setup()
{
  Serial.begin(57600);
  Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output
  // or the SD library functions will not work.
   pinMode(53, OUTPUT);
   
  if (!SD.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
 
  String filenamePre = "fr";
  String filenamePost = ".txt";
  String filenameStr = "fr21.txt";
  char filename[13]; 
  char testBuff[513];
 
 
  Serial.println("beginning read");
  Serial.println(millis());
 
  int j = 0;
  while(j < 30)
  {
 
    // re-open the file for reading:
    filenameStr = filenamePre + j + filenamePost;
    filenameStr.toCharArray(filename, 13);
    myFile = SD.open(filename);
    if (myFile) {
      //Serial.println(filename);
     
      // read from the file until there's nothing else in it:
      int byteCount = 0;
      while (myFile.available())
      {
        testBuff[byteCount] = myFile.read();
        byteCount++;
      }
      testBuff[byteCount] = NULL;

     
      // close the file:
      myFile.close();
    } else {
    // if the file didn't open, print an error:
      Serial.println("error opening test.txt");
    }
   
   
   // Serial.println(testBuff);
   
  j++;
  }
 
  Serial.println();
  Serial.println(millis());
 
 
 
}

void loop()
{
// nothing happens after setup
}
« Last Edit: June 06, 2012, 06:34:39 am by mrboni » Logged

Pages: [1] 2   Go Up
Jump to: