(Solved)-Reading SD-Card data through TCP (Ethernet Shield + SD-Card)

Hello everyone,

I’m new with Arduino, and I’m trying to read data stored on a .txt file on SD-Card and sen it to a TCP Client.
I’m using my Arduino as TCP Server.

The data I have on my SD-Card is on this format:
18434835
18434836
18434837

On my software TCP-Client (Computer side) I receive this data with some junk like this:
18434835 ˾Lu
18434846 ˾Lu
18434857 ˾Lu

Also I’m trying to read about 2000 of lines on this txt file, but I don’t know why I just receive about 100 to 150 lines max. and it’s just stop.
Note that another strange thing is that it’s jumping some lines (the number ends with 35,36,37 … and I receive 35,46,57…)

Here is my code:

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

File myFile;
char buf[10];

// Enter a MAC address, IP address and Portnumber for your Server below.
// The IP address will be dependent on your local network:
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress serverIP(10,190,24,76);
int serverPort=8082;

// Initialize the Ethernet server library
// with the IP address and port you want to use
EthernetServer server(serverPort);

void setup()
{
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  if(SD.begin(4) == 0) Serial.println("SD fail");
  else Serial.println("SD ok");

  Ethernet.begin(mac,serverIP);
  // Ethernet.begin() returns with its SPI enabled, so you must disable it.
  digitalWrite(10,HIGH);

  Serial.println("Ready");
}

void loop()
{
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    String clientMsg ="";
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        //Serial.print(c);
        clientMsg+=c;//store the recieved chracters in a string
        //if the character is an "end of line" the whole message is recieved
        if (c == '\n') {          
          // Open the file for reading:
          myFile = SD.open("test.txt");
          if (myFile) {
            Serial.println("test.txt:");
            
            // read from the file until there's nothing else in it:
            while (myFile.available()) {            	
                myFile.read(buf,10);                
                if (myFile.read() == '\n') {                  
                  client.println(buf);
                }
            }
            // close the file:
            myFile.close();
          } else {
          	// if the file didn't open, print an error:
            Serial.println("error opening test.txt");
          }//End SD Reading
        }
      }
    }
    // give the Client time to receive the data
    delay(1);
    // close the connection:
    client.stop();
  }
}
            while (myFile.available()) {                
                myFile.read(buf,10);                
                if (myFile.read() == '\n') {                  
                  client.println(buf);
                }
            }

As long as there is data to read, read 10 characters, and store them in an array that can hold 10 characters. Then, pass that non-NULL terminated array to a function that expects a NULL terminated array, and you get garbage out.

Make your array one element larger. Capture the return value from File::read() (the number of bytes actually read), and use that value as an array index, and store a NULL at that location. No more garbage will be sent.

Hello PaulS, thanks for your help!

Could you help me with this code ? how to do that you told by code ?

Thank you very much!

Could you help me with this code ? how to do that you told by code ?

Which part?

Make your array one element larger.

This could hardly be simpler: char buf[10]; --> char buf[11];

Capture the return value from File::read()

myFile.read(buf,10); --> int charCnt = myFile.read(buf,10);

and use that value as an array index, and store a NULL at that location.

buf[charCnt] = '\0';

Hello PauS, it’s me again!
Could you help me again with this ?

Now, I’m facing a problem that when I start reading the data on my SD-Card, almost time I get the “perfect string” like this on my TCP Client: “18432050” , but some times I don’t know why I get something like this: “184320451843204618432047184320481843204918432050”
I tried to increase the delay time on my while loop when read the data from SD, and I still facing this problem… if I decrease the delay It’s happens more frequently…
Here is my while loop:

while (myFile.available()) {
                if (!client.available()) {
                  break;
                }
                int charCnt = myFile.read(buf,9);                 
                delay(100);               
                if (myFile.read() == '\n') {                  
                  client.print(buf);                   
                }
                buf[charCnt] = '\0';                 
            }

Here is my entire code:

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>

File myFile;
char buf[10];

// Enter a MAC address, IP address and Portnumber for your Server below.
// The IP address will be dependent on your local network:
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
//IPAddress serverIP(192,168,2,130);
IPAddress serverIP(10,190,24,80);
int serverPort=8082;

// Initialize the Ethernet server library
// with the IP address and port you want to use
EthernetServer server(serverPort);

void setup()
{
  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);

  if(SD.begin(4) == 0) Serial.println("SD fail");
  else Serial.println("SD ok");

  Ethernet.begin(mac,serverIP);
  // Ethernet.begin() returns with its SPI enabled, so you must disable it.
  digitalWrite(10,HIGH);

  Serial.println("Ready");
}

void loop()
{
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    String clientMsg ="";
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        //Serial.print(c);
        clientMsg+=c;//store the recieved chracters in a string
        //if the character is an "end of line" the whole message is recieved
        if (c == 'r') {          
          // Open the file for reading:
          myFile = SD.open("test.txt");
          if (myFile) {
            Serial.println("test.txt:");
            
            // read from the file until there's nothing else in it:
            while (myFile.available()) {
                if (!client.available()) {
                  break;
                }
                int charCnt = myFile.read(buf,9);                 
                delay(100);               
                if (myFile.read() == '\n') {                  
                  client.print(buf);                   
                }
                buf[charCnt] = '\0';                 
            }
            // close the file:
            myFile.close();
            client.println("End of File Transmission!");
          } else {
          	// if the file didn't open, print an error:
            Serial.println("error opening test.txt");
          }//End SD Reading
        }
      }
    }
    // give the Client time to receive the data
    delay(5); 
    // close the connection:
    client.stop();   
      
  }
}

I tried to increase the delay time on my while loop when read the data from SD, and I still facing this problem... if I decrease the delay It's happens more frequently...

You don't need a delay in there at all.

How was the file created? If a function like File::println() was used to write to the file, the 8 character string is followed by a carriage return AND a line feed. You are only stripping out the carriage return, leaving the line feed.

How was the file created? If a function like File::println() was used to write to the file, the 8 character string is followed by a carriage return AND a line feed. You are only stripping out the carriage return, leaving the line feed.

Actually to perform a test, I just copy all the data from Excel file and past on the .txt file, but I'll use the myFile.println to write to the file! I'll write the code to write the file from Arduino and perform a test !

Just taking advantage, I'm tryind to send from TCP client requests like "Read" / " Write", in the code I used just "r" and I'll put "w"... How can I change the follow part of arduino code to receive the entore string like "Read" "Write" istead "r" or "w" on this:

if (c == 'r') {          
          // Open the file for reading:

Thanks Again !

How can I change the follow part of arduino code to receive the entore string like "Read" "Write" istead "r" or "w" on this:

Nothing in that snippet has anything to do with receiving anything. As I pointed out in your other thread, a char array and strcmp() are needed.

Nothing in that snippet has anything to do with receiving anything. As I pointed out in your other thread, a char array and strcmp() are needed.

Sorry for my Stupid question, but when I try to use the array, I receive the message: “invalid conversion from ‘char*’ to ‘uint8_t*’”

I have declared

char rec[10];

and I receive the error message on this line:

while (client.connected()) {
      if (client.available()) {       
        
        int charCount = client.read(rec,9);
...
...

Sorry for my Stupid question, but when I try to use the array, I receive the message: “invalid conversion from ‘char*’ to ‘uint8_t*’”

Either change the type to uint8_t (from char) or use a cast.

int charCount = client.read((uint8_t *)rec, 9);

Hello PauS, sorry to take your time again...

Now I can write to the txt file fine from my windows app TCP client... It's works fine if I send a string like "12345678" per secont, I can't speed up that... I have declared:

char rec[12];

And here is the code to write to the file:

 while (client.connected()) {
      if (client.available()) {        
        int charCount = client.read((uint8_t *)rec, 11);                 
          // open the file. note that only one file can be open at a time,
          // so you have to close this one before opening another.
          myFile = SD.open("abc.txt", FILE_WRITE);
          
          // if the file opened okay, write to it:
          if (myFile) {
            Serial.print("Writing to abc.txt...");
            myFile.print(rec);
            Serial.print(rec);
            // close the file:
            myFile.close();
            Serial.println("done.");
          } else {
            // if the file didn't open, print an error:
            Serial.println("error opening test.txt");
          }

If I try to speedup the write process, and I open the txt file to check ou, the format looks like this:

12345617
12345617
12345617
1345617

112345617
112345617
12345617
1345617

112345617
112345617

Do you know why and how to fix it ? Thanks Again!

Do you know why and how to fix it ?

Yes. It is happening because the delay() causes the value returned by client.available() to change, since more time is allowed for data to arrive.

The data that the server is sending needs to be packeted. That is, there needs to be (and may already be) something that defines the end of a value. You might have the server append a ! after the value, or a carriage return and/or line feed (though it is quite possible that the CR/LF are already appended).

Then, you simply need to read whatever is ready to be read, appending it to an array, until the end of packet marker arrives. When that happens, open the file, write the data, close the file, reset the index and NULL the array.

PaulS:
Yes. It is happening because the delay() causes the value returned by client.available() to change, since more time is allowed for data to arrive.

The data that the server is sending needs to be packeted. That is, there needs to be (and may already be) something that defines the end of a value. You might have the server append a ! after the value, or a carriage return and/or line feed (though it is quite possible that the CR/LF are already appended).

Then, you simply need to read whatever is ready to be read, appending it to an array, until the end of packet marker arrives. When that happens, open the file, write the data, close the file, reset the index and NULL the array.

Thanks PaulS for your repply!
Could you please help me how do to that ? I mean the code ? I still learning C/C++ language!
Thank you again !

This code reads data from the serial port that is between < and > markers. It is easy to change this to ignore the start marker (<) and to use a different end marker (or two) (\n and/or \r) and to read from client instead of Serial.

#define SOP '<'
#define EOP '>'

bool started = false;
bool ended = false;

char inData[80];
byte index;

void setup()
{
   Serial.begin(57600);
   // Other stuff...
}

void loop()
{
  // Read all serial data available, as fast as possible
  while(Serial.available() > 0)
  {
    char inChar = Serial.read();
    if(inChar == SOP)
    {
       index = 0;
       inData[index] = '\0';
       started = true;
       ended = false;
    }
    else if(inChar == EOP)
    {
       ended = true;
       break;
    }
    else
    {
      if(index < 79)
      {
        inData[index] = inChar;
        index++;
        inData[index] = '\0';
      }
    }
  }

  // We are here either because all pending serial
  // data has been read OR because an end of
  // packet marker arrived. Which is it?
  if(started && ended)
  {
    // The end of packet marker arrived. Process the packet

    // Reset for the next packet
    started = false;
    ended = false;
    index = 0;
    inData[index] = '\0';
  }
}

You already have the code to open the file, etc.

PaulS, I just perform a test and it's works verry well !

Thank you for your time and your help!

best regards, Rodrigo.