Passing Arduino Data to Webserver

Hi y'all ^_^

I'm starting a new project and I was hoping to run this by the community first and get some hints. The general idea is to log sensor data from an Arduino with an Ethernet shield (possibly with SD card) to an Apache web-server. The raw sensor data would be converted into graphs and real time values (using a php script?) and would be accessible to anyone in the network through their browser. The system would be running 24/7 observing an Aquaponics setup. The goal is to have both an historic archive of sensor data for research, and an overview of real time sensor values.

It seems there are a number of different ways of achieving what I am trying to do. I'm a quick learner and not intimidated by having to figure out some new things, but I also want to avoid getting stuck in dead ends, wasting time trying to make some god awful hack work reliably when I should have gone a different route entirely.

So here is my question: What has worked for you? How would you go about achieving the kind of result that I am looking for?

Thank you for your collaboration :) !

How would you go about achieving the kind of result that I am looking for?

It isn't all that clear what results you are looking for.

I would set up a web server that the Arduino as client could access, so that the Arduino could have the server store some data.

Then, I'd have other scripts that accessed the same data that the Arduino as client sent, for other clients to access, that would get data from the same repository, most like MySQL, and have those scripts do the data display as graphs.

Thank you for your response!

PaulS:
It isn’t all that clear what results you are looking for.

Oh, sorry about that. Let me try again: The result I am looking for is a graphical representation of an archive of historic sensor data for research, and an overview of real time sensor values for monitoring, accessible by internet browser on our LAN.

What I’m working with: Arduino 2009, Ethernet Shield, and a dedicated Ubuntu server with LAMP. I haven’t quite done anything like this before, hence trying to get some pointers from people with more experience with this sort of thing, what sort of challenges they encountered in their approach etc.

I haven’t even thought about using an actual database to store the data, I was just thinking maybe parsing values that are being written into a txt file, but I can totally see the advantages of writing to a database and being able to use the data for all kinds of things from there on out. Ultimately I’m hoping to be able to add any number of Arduino clients to the network, so we can gather all kinds of environmental data, and storing it in a database would allow us to cross reference and compare our sensor data in all sorts of interesting ways. Thanks for the idea!

Below is simple server code to supply updating arduino analog pin values to a browser. To store data, you could store it in the main server as a comma delinated file viewable in excel and such.

// zoomkat meta refresh server test code
// arduino IDE 1.0
// for W5100 ethernet shield
// the IP address will be dependent on your local network/router
// port 80 is default for HTTP, but can be changed as needed
// use IP address like http://192.168.1.102:84 in your brouser
// or http://zoomkat.no-ip.com:84 with dynamic IP service
// use the \ slash to escape the " in the html
// meta refresh set for 2 seconds

#include <SPI.h>
#include <Ethernet.h>

int x=0; //set refresh counter to 0
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(192,168,1,102); // ip in lan
EthernetServer server(84); //server is using port 84

void setup()
{
  // start the server
  Ethernet.begin(mac, ip);
  server.begin();
}

void loop()
{
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
     while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        // see if HTTP request has ended with blank line
        if (c == '\n') {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println();
          
          //meta-refresh page every 2 seconds
          x=x+1; //page upload counter
          client.println("<HTML>");
          client.print("<HEAD>");
          client.print("<meta http-equiv=\"refresh\" content=\"2\">");
          client.print("<TITLE />Zoomkat's meta-refresh test</title>");
          client.print("</head>");
          client.println("<BODY>");
          client.print("Zoomkat's meta-refresh test IDE 1.0");
          client.println("
");
                    
          client.print("page refresh number ");
          client.println(x); //current refresh count
          client.println("
");
          client.println("
");
          
          client.print("Zoomkat's arduino analog input values:");
          client.println("
");
          client.println("
");
          
          // output the value of each analog input pin
          for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(analogRead(analogChannel));
            client.println("
");
            }
           break;
          client.println("</BODY>");
          client.println("</HTML>");
         }
        }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
  }
}

zoomkat: To store data, you could store it in the main server as a comma delinated file viewable in excel and such.

I recommend using a relational database rather than a flat file. It will be more resilient, enable concurrent access, enable retrospective analysis/summarisation/garbage collection etc, and give you the option to extend the data model in arbitrary ways in future. Given a LAMP environment, it is imo a no-brainer.

I recommend using a relational database rather than a flat file.

I'd agree. But, I think that zoomkat's intent was to show how to get data from the Arduino to a server. The idea of storing the data in a flat file was probably just meant to show a way of enabling that the script on the server was indeed being called, and called with the right data, and that the script was parsing the data correctly.

Once those three things are proven, changing how the data is persisted, from a flat file to a relational database is trivial.

Great, those are some pretty solid leads. I can work a strategy off of that. Thanks for all the advice!