Go Down

Topic: reading a txt file from an sd card over the internet (Read 2008 times) previous topic - next topic

I have my uno set up to take a temperature measurement every thirty minutes and log the data into an sd card. ( I am using the arduino Ethernet board W5100). I want to access the information stored in the log file through the internet. Given that I can't access both Ethernet and sd at the same time, what would be the best way to do this? Should I save the data to an array or string then pass that on to the Ethernet. seems like that defeats the purpose of having external storage. any other examples or threads would be appreciated. Sorry if this has been asked a hundred times before.

pYro_65

#1
Jul 05, 2012, 03:41 am Last Edit: Jul 05, 2012, 03:45 am by pYro_65 Reason: 1
I would personally use a web based server, then have the arduino upload periodically.

But with the arduino as the main interface, you could send out a header then when its time for the data, switch between sd and ethernet in a loop reading and sending a small chunk of data.

As the arduino is the server, there may be ways to increase the maximum server time-out if sd access becomes lengthy.

Quote
seems like that defeats the purpose of having external storage


The sd is a completely independent device, as the Ethernet shield is used on micros that have limited resources, the creator has outweighed cost over features. Its pretty much a bonus, as many Ethernet setups will need the additional space.

PaulS

Quote
Given that I can't access both Ethernet and sd at the same time

The Arduino libraries that get data from/send data to the Ethernet card or the SD card manage the switching fro you.

Given that, your assumption is invalid. The Arduino CAN access both devices, though, technically, not at the same time. The access is sequential, but fast enough that it doesn't matter that it is not "at the same time".


nice to know I misunderstood that. so how would I go about doing this? initialize one at a time, then do something like :  client.print(myFile.read()); ? do you know where I could find a good example of how to combine data logging and a webserver? thanks

Here is the code I pieced together. Everything works, except the part where I am actually able to read the  whole file over the internet.
Code: [Select]

#include <SPI.h>
#include <Ethernet.h>
#include <OneWire.h>
#include <SD.h>
int DS18S20_Pin = 9;
OneWire ds(DS18S20_Pin);
const int sd_cs = 4;
const int ethernet_cs = 10;
long interval = 200000;
long previousMillis = 0;
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1,25);


EthernetServer server(80);
File myFile;
void setup() {

  Serial.begin(9600);
pinMode(sd_cs, OUTPUT);
  pinMode(ethernet_cs, OUTPUT);
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
  digitalWrite(ethernet_cs, HIGH);
  digitalWrite(sd_cs, LOW);
  if (!SD.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
  myFile = SD.open("temps.txt", FILE_WRITE);
  myFile.close();
}


void loop() {
  checkClient();
  unsigned long currentMillis = millis();
  if(currentMillis - previousMillis > interval) {
    previousMillis = currentMillis;   
    recordConditions();
  }
}

void checkClient()  // listen for incoming clients
{
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
       
        if (c == '\n' && currentLineIsBlank) {
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connnection: close");
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          client.println("<meta http-equiv=\"refresh\" content=\"5\">");
          char b = getFile();
          client.print(b);
          client.println(" done");
          Serial.println(" done");
          client.println("<br />");       
          client.println("</html>");
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        }
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
   
    delay(1);
   
    client.stop();
    Serial.println("client disonnected");
  }
}
char getFile(){
  myFile = SD.open("temps.txt");
  if (myFile) {
    Serial.println("temps.txt:");
    while (myFile.available()) {
      Serial.write(myFile.read());
      char a = myFile.read();
      return a;

    }
   
    myFile.close();
  }
  else {
   Serial.println("error opening test.txt");
  }
}
void recordConditions()
{
  myFile = SD.open("temps.txt", FILE_WRITE);
   if (myFile) {
    Serial.print("Writing to temp.txt...");
    float temperature = getTemp();
    Serial.print(temperature);
    myFile.println(temperature);
    myFile.close();
    Serial.println("done.");
  }
  else {
    Serial.println("error opening temps.txt");
  }
}
float getTemp(){
  byte data[12];
  byte addr[8];

  if ( !ds.search(addr)) {
    ds.reset_search();
    return -1000;
  }

  if ( OneWire::crc8( addr, 7) != addr[7]) {
    Serial.print("CRC is not valid!\n");
    return -1000;
  }

  if ( addr[0] != 0x10 && addr[0] != 0x28) {
    Serial.print("Device is not recognized");
    return -1000;
  }

  ds.reset();
  ds.select(addr);
  ds.write(0x44,1);

  byte present = ds.reset();
  ds.select(addr);   
  ds.write(0xBE);


  for (int i = 0; i < 9; i++) {
    data[i] = ds.read();
  }

  ds.reset_search();

  byte MSB = data[1];
  byte LSB = data[0];

  float tempRead = ((MSB << 8) | LSB);
  float TemperatureSum = tempRead / 16 * 9 / 5 + 32;

  return TemperatureSum;

}



Important part of the output in the serial monitor looks like this:
Quote

temps.txt:
7 done
client disonnected
Writing to temp.txt...77.11done.
Writing to temp.txt...77.00done.

This points my attention to the getFile() call which prints the "temps.txt:" above and then its supposed to print the whole file followed by "done". The problem is I am only getting one character. The same thing happens over the internet but half the time the one character sent is completely random. I think I did something stupid somewhere. Do you have any suggestions, especially for how to read the file then print the information to ethernet client. The way I am trying to do it just doesn't seem right. 

PaulS

Quote
initialize one at a time, then do something like :  client.print(myFile.read()); ?

In that statement, you are NOT switching which device is active. You would need to do something like:

Code: [Select]
// Make SD active
char ltr = myFile.read();
// Make Ethernet active
client.print(ltr);

(where you put real code to activate the two devices, not comments like I did).

Quote
This points my attention to the getFile() call which prints the "temps.txt:" above and then its supposed to print the whole file followed by "done". The problem is I am only getting one character.

That is all that code is supposed to do. You open the file. read and print one character, and read and return the next character, The return statement ends the getFile() function, which causes reading to stop.

Doing two reads, like that, is not a good idea.

#6
Jul 07, 2012, 06:45 am Last Edit: Jul 07, 2012, 06:48 am by seanz2003 Reason: 1
thank you PaulS for your reply!

Quote
The return statement ends the getFile() function, which causes reading to stop.


So if i want to read the whole file, I cannot use return? Could you explain how  " char a = myFile.read(); " works? In a learning example, I used " Serial.write(myFile.read()); " to print out the entire content of a file, whereas here i am only getting one character. how is it these two statements yield different results? In my mind, it seems that  " char a = myFile.read(); " should read the entire file into char a...
..... Hmm .. I just read up on the char data type and i don't think it is appropriate at all if I want to read out 100+ temperature readings.  So i'll try a string then (or is it String?).

PaulS

Quote
So if i want to read the whole file, I cannot use return?

Sure you can. Just not there.

What is it the function should return? You have it defined to return 1 character, which really does not make sense.

If the intent is to return the contents of the file, that isn't really possible. You could pass another variable to the function, by reference, that is an array where the data would be stored, making the function return type void, and eliminating the need for a return statement.

Quote
In a learning example, I used " Serial.write(myFile.read()); " to print out the entire content of a file

That was done in a while loop, one character at a time.

Quote
So i'll try a string then (or is it String?).

string - a NULL terminated array of chars.
String - a class that will almost certainly fragment all of your memory, causing your Arduino to crash.

You decide which to use.

Is this better: char dataString [] = "myFile.read()"; ?

ah, Working code ! Thanks for your help PaulS! I learned something tonight.

Code: [Select]
#include <SPI.h>
#include <Ethernet.h>
#include <OneWire.h>
#include <SD.h>
int DS18S20_Pin = 9;
OneWire ds(DS18S20_Pin);
const int sd_cs = 4;
const int ethernet_cs = 10;
long interval = 200000;
long previousMillis = 0;
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1,25);

// Initialize the Ethernet server library
// with the IP address and port you want to use
// (port 80 is default for HTTP):
EthernetServer server(80);
File myFile;
char data [] = "";
void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);

  pinMode(sd_cs, OUTPUT);
  pinMode(ethernet_cs, OUTPUT);
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
  digitalWrite(ethernet_cs, HIGH);
  digitalWrite(sd_cs, LOW);
  if (!SD.begin(4)) {
    Serial.println("initialization failed!");
    return;
  }
  Serial.println("initialization done.");
  myFile = SD.open("temps.txt", FILE_WRITE);
  myFile.close();
}


void loop() {

  checkClient();
  unsigned long currentMillis = millis();

  if(currentMillis - previousMillis > interval) {

    previousMillis = currentMillis;   
    recordConditions();

  }

}

void checkClient()  // listen for incoming clients
{
  digitalWrite(ethernet_cs, LOW);
  digitalWrite(sd_cs, HIGH);
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == '\n' && currentLineIsBlank) {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connnection: close");
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");
          // add a meta refresh tag, so the browser pulls again every 5 seconds:
          client.println("<meta http-equiv=\"refresh\" content=\"5\">");
          digitalWrite(ethernet_cs, HIGH);
          digitalWrite(sd_cs, LOW);
          myFile = SD.open("temps.txt");
          if (myFile) {
            Serial.println("temps.txt:");

            // read from the file until there's nothing else in it:
            while (myFile.available()) {

              char data = myFile.read();

              digitalWrite(ethernet_cs, LOW);
              digitalWrite(sd_cs, HIGH);
              client.print(data);
              Serial.write(data); 

            }
            // close the file:
            digitalWrite(ethernet_cs, HIGH);
            digitalWrite(sd_cs, LOW);
            myFile.close();
          }
          else {
            // if the file didn't open, print an error:
            Serial.println("error opening test.txt");
          }
          digitalWrite(ethernet_cs, LOW);
          digitalWrite(sd_cs, HIGH);
          client.println(" done");
          Serial.println(" done");
          client.println("<br />");       
          client.println("</html>");
          break;
        }
        if (c == '\n') {
          // you're starting a new line
          currentLineIsBlank = true;
        }
        else if (c != '\r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}
void recordConditions()
{
  digitalWrite(ethernet_cs, HIGH);
  digitalWrite(sd_cs, LOW);
  myFile = SD.open("temps.txt", FILE_WRITE);

  // if the file opened okay, write to it:
  if (myFile) {
    Serial.print("Writing to temp.txt...");
    float temperature = getTemp();
    Serial.print(temperature);
    myFile.println(temperature);
    // close the file:
    myFile.close();
    Serial.println("done.");
  }
  else {
    // if the file didn't open, print an error:
    Serial.println("error opening temps.txt");
  }
}
float getTemp(){
  //returns the temperature from one DS18S20 in DEG Celsius

  byte data[12];
  byte addr[8];

  if ( !ds.search(addr)) {
    //no more sensors on chain, reset search
    ds.reset_search();
    return -1000;
  }

  if ( OneWire::crc8( addr, 7) != addr[7]) {
    Serial.print("CRC is not valid!\n");
    return -1000;
  }

  if ( addr[0] != 0x10 && addr[0] != 0x28) {
    Serial.print("Device is not recognized");
    return -1000;
  }

  ds.reset();
  ds.select(addr);
  ds.write(0x44,1); // start conversion, with parasite power on at the end

  byte present = ds.reset();
  ds.select(addr);   
  ds.write(0xBE); // Read Scratchpad


  for (int i = 0; i < 9; i++) { // we need 9 bytes
    data[i] = ds.read();
  }

  ds.reset_search();

  byte MSB = data[1];
  byte LSB = data[0];

  float tempRead = ((MSB << 8) | LSB); //using two's compliment
  float TemperatureSum = tempRead / 16 * 9 / 5 + 32;

  return TemperatureSum;

}

PaulS

Code: [Select]
  myFile = SD.open("temps.txt", FILE_WRITE);
  myFile.close();

What is this accomplishing?

The next step would be to see if those statements to explicitly toggle the active SPI device are really needed. I have code that reads SD card data and returns it to the client and I don't need to explicitly manage the active chip select pin.

After that, creating some functions would be something to learn how to do. The block of code that reads the SD file and sends it to the client would be the first stuff I'd move into a function. Look for places where a reasonable amount of code is doing one thing and doing that one thing well, and put that code in a function.

Just a comment, that you are free to ignore. Functions should have names that clearly indicate what they do. A good function name will even imply what it will return. Your checkClient() function implies that it is going to check whether there is a client connection to deal with, or not. It implies that it is going to return either a true/false value (there is, or is not a client connection to deal with) or a client object that needs to be dealt with.

A name like dealWithClients() would, instead, imply that the function will determine if there are any clients to be dealt with, and deal with them. Of course, deal is a rather vague verb to be using, since there is nothing in the word that says how the client should be dealt with. Pissing the client off so they never return is one way of dealing with them. Probably not the best way.

The point is to create a name for a function that leaves no ambiguity about that the function does. The recordConditions() name is an example of such a function name.


Go Up