Automatic reading of arduino values WEBSERVER

Hello! I'm having difficulties to make the html page stored in the Ethernet shield SD card, and loaded into browser, to permanently receive the values of my ultrasonic sensor, and change the height of an image (make a dynamic image), using only arduino+ethernet shield+SD card and my intranet (independant system of internet, computers,etc).

Thanks!!

Is it possible to host an php page on SD card of ethernet shield and when this page is loaded by IP (of the ethernet shield), the arduino keep sending sensor value to this loaded page without meta refresh tag? In other words... keep arduino connected to the page in real time, and who is seeing the page see the values changing in real time?

blastboot: Is it possible to host an php page on SD card of ethernet shield and when this page is loaded by IP (of the ethernet shield), the arduino keep sending sensor value to this loaded page without meta refresh tag? In other words... keep arduino connected to the page in real time, and who is seeing the page see the values changing in real time?

You should be able to test the PHP question yourself. As to the meta refresh data, put it in a frame on your main page.

I've had tested it before but if the webpage hosted in sd card has some images, the meta refresh makes it slow to refresh because arduino has to load all images everythime, and i only want it to refresh a specific place (sensor values). There's any solution to do this?

I’ve had tested it before but if the webpage hosted in sd card has some images, the meta refresh makes it slow to refresh because arduino has to load all images everythime, and i only want it to refresh a specific place (sensor values). There’s any solution to do this?

Sure, “As to the meta refresh data, put it in a frame on your main page.” No need to reload the images.

Sounds like you are wanting to display a HTML page which displays something like an image that changes in height as the value of your ultrasonic changes, is my understanding correct ?

Also, you are wanting to do this without using any other devices, such as servers on a hosted site, just using the Arduino and router to connect it to the internet, is this correct ?

If this is correct, how about this for an idea.

Firstly, forget about using php, you will need a server that has php, and the Arduino doesn't, can't support that. Next, instead of using an image would it be a better idea to use HTML5 and use the 'canvas' element and then use HTML stroke and fill commands to draw your shape, say a rectangle. The rectangle would be sized according to your value from your ultrasonic sensor.

I imagine this would be quite a small and simple html document.

You could also look at using jQuery and have the client side request the data from the Arduino typically in json format every few seconds as you wish. Using this concept is very neat and allows you to expand upon your idea as your program starts to grow.

Have a look at something I am working on in my spare time. I have an Arduino Mega with on board ethernet and SD card connected to my router. The client side request data directly from the Arduino using json format. Also, the Arduino pushes data up to the host server where I have a php file take that and store in a SQL database. It is still a work in progress at this point in time.

But it might give you some ideas of what you can do.

Go to http://paulalting.com/hydrosolar.php

Regards, Paul

You're right! That's exactaly what i want to do. I think i can use PHP because in this intranet there's servers with PHP (Apache) running so they can "decode" it. My problem is that i only want to use intranet to load the page (stored in SD card) in browser and this page receive sensor values and then modify an image height (only from bottom to the top direction). So i will read the link you posted and search for information you give, meanwhile i keep not understanding if an webpage loaded from arduino has the capability to "act by its own" (modify image with sensor value every 2 minutes, for exemple). Thanks for helping me!

Very cool Paul, Are you using Rgraph for the dials, or your own code or ??

I'm pushing data to KSduino.com and he has graphs like this running: http://ksduino.org/?devices&device_parameter_id=194 Main data here: http://ksduino.org/?devices&device_id=3330

But I'd like to learn how to use HTML5 Canvas myself to create "Control Panels".. Total Newbie on that!

Maybe you could start a thread on this subject???

If it's control panels in HTML5 you want, take a look at Bitlash Commander: https://github.com/billroy/bitlash-commander

Commander is intended to work with USB-connected Arduinos, but it is happy to accept HTTP POST data from ethernet-connected Arduinos as well.

For a simpler example there's ArduinoLiveChart: https://github.com/billroy/arduino-livechart

Not a drop-in solution for the OP's request but may be some useful ideas there.

Best,

-br

Hello there Terry,
Nice to chat again, thank you for your comments.

The dials are created in HTML5 with the various stroke commands, though I did not make them, but simply use them in the javascript.
The historical trending charts are done using Highcharts, (a lot of work to be done here yet).
I shall put together some info in the coming days then. But I am doing a lot of work on the offline web site at present which has a lot of information about the whole system. There are some issues with the javascript as it is on the server at present, I have some fixes to upload to the server tomorrow hopefully.

Also, note that if you use Internet Explorer < 9, you may get strange or no results. I run OSX and Ubuntu, so Firefox, Chrome or Safari.

Yes, I looked at your KSduino graphs today, they look like they work well, and probably without a lot of the pain and hair pulling I am going through. It is fun and I am using it to learn more.

Good idea about a new thread, I would be happy to participate with that.

P.S. remember I had my Arduino Mega fries by lightning, well I was happy to receive free replacements for both the fried Ether-Mega and Ether-Ten from Freetronics.

Sleep time now in Australia, 12:25am
Kind regards,
Paul

Bill, That demo chart looks good. I see you use Node.js, which is something I looked into some weeks ago, it's very fast. Though I wasn't sure if I could put Node.js up on my hosted site.

Will look more tomorrow, Paul

FWIW, Most vanilla hosting hasn't caught up to node.js yet, but you can host small node.js apps for free on Heroku: http://www.heroku.com among other places (NodeJitsu and AppFog, for example).

-br

The projects you're been posting are possible to do with the webpage stored in the ethernet shield SD Card ??

rockwallaby The project of hydrosolar.php it's excellent and has one thing that i wanted to do in mine, is that the values changing without reload the image. Thanks for helping me

blastboot wrote: The projects you're been posting are possible to do with the web page stored in the ethernet shield SD Card ??

Yes, it is. In fact my project started out this way, only using the Arduino itself. But there are cautions of course. With the Arduino SD card, care is needed around programing when it comes to having the same system work with ethernet. I believe the SD card system is relatively slow also, which may impact on how and what you want to do. Keep in mind the idea that you can serve up a simple html file from the SD card and have your html code point to external resources such as fancy stuff like CSS or external libraries. Doing this will help with performance as the browser will get those files from high speed servers.

The Arduino Mega I use has onboard SD card slot as well as wiznet 5100 ethernet controller, so I simply connect it to a power supply and with a cat 5 cable, to my router.

But to remotely access your Arduino, you need to know your external IP address. I knew the situation with mine, as it is static, it does not change over time.

Depending on your internet service provider, you may have either a static or dynamic IP assigned. If dynamic you will need to use a service like 'no-ip', or 'dyndns' to help you with this. Maybe your router has the provision to do this also.

Remember for remote access of any sort you will need to connect your Arduino to your ADSL/DSL router which is then connected to the internet via your ISP.

As far as the program is concerned, there are a few ways to do this, and Bill's way would be quite a suitable solution I think from what I see of Bitlash. I will need to let Bill say if it would work with what you are wanting to do.

I would start with something very simple and build up from there.

Have the Arduino reliably read the ultrasonic signal and confirm this firstly in your serial monitor in the IDE. Play around with this and get to know the limits and conditions under which ultrasonics will work reliably.

Then start with a section of program to present this data to a web browser if that is your intention. I played around initially with some of the example web server programs here on the Arduino site, but I ended up using Webduino library as I really liked what it is able to do, as well as providing a good working framework that part of the software.

So, just checking, you do want to remotely view the ultrasonic level that your Arduino is reading? And that you wish to use the Arduino to present this information as a dynamically updating html page?

If all the above is correct, can you tell us what you have done so far in terms of the program for the Arduino or are you just at the beginning stage?

Regards, Paul

At this time I have made a simple HTML like you can see and i have this code on arduino that you can see below.
My problem is to make an animation inside the water tank that simulates water level according the ultrasonic values.
Another problem is to refresh values (% and Lt) without refreshing the whole page, because arduino is a little bit slower to upload images in browser. Keep in mind that i’m working on a Intranet and i don’t have access to internet and the system has to be independant (no server storage files, only ethernet shield SD card).
I’m portugese so i’ve made some portuguese comments on program. This arduino program was searched on internet and it was a little adapted to survey my project. I’m trying to make an iframe like zoomkat said. Thanks for helping me XD

/* SETUP CARTÃO SD: O SD DEVE ESTAR FORMATADO EM "FAT" E DEVE CONTER UM FICHEIRO
 * COM O NOME "index.htm". TODOS OS FICHEIROS NO CARTÃO DEVEM TER NOMES COM NO
 * MÁXIMO 8 CARACTERES E EXTENSÕES COM NO MÁXIMO 3 CARACTERES.
 */
 
#include <SPI.h>
#include <SdFat.h>
#include <SdFatUtil.h>
#include <Ethernet.h>
#define greenLEDandBEEP 2

//O BUFFER DEVE SER GRANDE
#define BUFSIZ 128

float leitura = 0;
float percentagem = 0;
float litros = 0;
char buffer[3];

//CONFIGURAÇÃO DO ETHERNET SHIELD

byte mac[] = { 
  0x90, 0xA2, 0xDA, 0x00, 0xEA, 0x8C };
IPAddress ip(196,168,0,3);
char rootFileName[] = "index.htm";
EthernetServer server(80);

//PROGRAMAÇÃO PARA O CARTÃO SD

Sd2Card card;
SdVolume volume;
SdFile root;
SdFile file;

//ARMAZENA SEQUÊNCIAS DE ERRO NA MEMÓRIA FLASH PARA ECONOMIZAR RAM

#define error(s) error_P(PSTR(s))
void error_P(const char* str) {
  PgmPrint("Erro: ");
  SerialPrintln_P(str);
  if (card.errorCode()) {
    PgmPrint("Erro cartao SD: ");
    Serial.print(card.errorCode(), HEX);
    Serial.print(',');
    Serial.println(card.errorData(), HEX);
  }
  while(1);
}

void setup() {
  
  Serial.begin(9600);
  
  pinMode(greenLEDandBEEP, OUTPUT);
 
  PgmPrint("Memoria RAM Livre: ");
  Serial.println(FreeRam());  
    
  pinMode(10, OUTPUT);              
  digitalWrite(10, HIGH);              

//INICIALIZA O CARTÃO SD EM "FULL SPEED", MÁXIMO DESEMPENHO. (PARA EVITAR ERROS COM LIGAÇÃO À BREADBOARD DEVE ESTAR EM "HALF SPEED"

  if (!card.init(SPI_FULL_SPEED, 4)) error("Falha no card.init!");

//INICIALIZA O VOLUME FAT

  if (!volume.init(&card)) error("Falha no vol.init!");

  PgmPrint("O volume e FAT");
  Serial.println(volume.fatType(),DEC);
  Serial.println();

  if (!root.openRoot(&volume)) error("Falha no openRoot!");

  PgmPrintln("Ficheiros na raiz do cartao:");
  
//LISTA NA SÉRIE OS FICHEIROS QUE ESTÃO NA RAIZ DO CARTÃO SD, COM DATA E TAMANHO
  
  root.ls(LS_DATE | LS_SIZE);
  Serial.println();

//LISTA OS FICHEIROS DE TODOS OS DIRECTÓRIOS DO CARTÃO SD

  PgmPrintln("Ficheiros encontrados em todos os directorios do cartao:");
  root.ls(LS_R);

  Serial.println();
  PgmPrintln("Concluido");
  
//CONCLUI A DEPURAÇÃO E INICIA O SERVIDOR  

  Ethernet.begin(mac, ip);
  server.begin();
  delay(1);
}


void loop()
{
  char clientline[BUFSIZ];
  char *filename;
  int image = 0;
  int index = 0;

  EthernetClient client = server.available();
  if (client) {
    boolean current_line_is_blank = true;

//REPÕE O BUFFER DE ENTRADA

    index = 0;

    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

//SE NÃO É UMA NOVA LINHA, GUARDA OS CARACTERES RECEBIDOS

        if (c != '\n' && c != '\r') {
          clientline[index] = c;
          index++;

//SE O BUFFER NÃO FOR GRANDE O SUFICIENTE, APAGA ALGUNS DADOS

          if (index >= BUFSIZ) 
            index = BUFSIZ -1;
          //CONTINUA A LER MAIS DADOS
          continue;
        }

//SE TEM \n OU \r ENTÃO A RECEPÇÃO DE DADOS ESTÁ CONCLUÍDA

        clientline[index] = 0;
        filename = 0;

//IMPRIME NA SÉRIE PARA DEPURAÇÃO

        Serial.println(clientline);

//PROCURA "GET / " PARA ENVIAR O index.htm

        if (strstr(clientline, "GET / ") != 0) {
          filename = rootFileName;

        } 
        if (strstr(clientline, "GET /") != 0) {

//SE NÃO HOUVER ESPAÇO A SEGUIR À "/", ENTÃO ESTÁ A PEDIR UM SUBDIRECTÓRIO OU FICHEIRO

          if (!filename) filename = clientline + 5; 

//VÊ OS 5 CARACTERES APÓS O "GET /"

          (strstr(clientline, " HTTP"))[0] = 0;

//IMPRIME O NOME DO FICHEIRO "index.htm" NA SÉRIE

          Serial.println(filename);

          if (! file.open(&root, filename, O_READ)) {
            client.println("HTTP/1.1 404 Not Found");
            client.println("Content-Type: text/html");
            client.println();
            client.println("<h2>Erro 404</h2>");
            client.println("<s2>O ficheiro nao existe.<s2>");
            client.println("");
            break;
          }

          Serial.println("Ligado!");

//TIPOS DE FICHEIRO QUE PODEM ESTAR NO CARTÃO SD

          client.println("HTTP/1.1 200 OK");
          if (strstr(filename, ".htm") != 0)
            client.println("Content-Type: text/html");
          else if (strstr(filename, ".css") != 0)
            client.println("Content-Type: text/css");
          else if (strstr(filename, ".png") != 0)
            client.println("Content-Type: text/php");
          else if (strstr(filename, ".php") != 0)
            client.println("Content-Type: image/png");
          else if (strstr(filename, ".jpg") != 0)
            client.println("Content-Type: image/jpeg");
          else if (strstr(filename, ".gif") != 0)
            client.println("Content-Type: image/gif");
          else if (strstr(filename, ".3gp") != 0)
            client.println("Content-Type: video/mpeg");
          else if (strstr(filename, ".pdf") != 0)
            client.println("Content-Type: application/pdf");
          else if (strstr(filename, ".js") != 0)
            client.println("Content-Type: application/x-javascript");
          else if (strstr(filename, ".xml") != 0)
            client.println("Content-Type: application/xml");
          else
            client.println("Content-Type: text");
          client.println();

          int16_t c;
          while ((c = file.read()) >= 0) {

//IMPRIME NA SÉRIE TODO O CÓDIGO HTML (LENTO!) POR ISSO ESTÁ DESACTIVADO
            
            /*  Serial.print((char)c);  */
            
//IMPRIME NO BROWSER A PÁGINA HTML (CÓDIGO DO FICHEIRO "index.htm")
            
            client.print((char)c);
          }

//A PARTIR DAQUI SERÁ CÓDIGO HTML A ACRESCENTAR AO "index.htm"
        
          client.print("<html><body>");
          //client.print("<P align=\"center\">"); 
          //client.print("Pagina elaborada por GRUPO 1 <b>");   
          //client.print("</b>
");
  
//LEITURA DOS DADOS DO SENSOR E CONVERSÃO PARA PERCENTAGEM

        if ( Serial.available () > 0 )
  {
    if (Serial.read() == 'R')
   {
     Serial.readBytesUntil ( '\r', &buffer[0], 3 );
  
  leitura = atoi(buffer);
  percentagem = (leitura*100)/465;
  
  Serial.print("Nivel do Tanque:");
  Serial.print(percentagem, 1);
  Serial.println("%");
 
 //CÁLCULOS DA QUANTIDADE DE ÀGUA NO TANQUE
          
  litros = (percentagem*700000)/100;        
  
  Serial.print("Quantidade de agua no Tanque:");
  Serial.print(litros, 1);
  Serial.println("litros");
}
  
}
          
      
       if (percentagem >= 90 || percentagem <=10){
          client.print("<p>Nivel do Tanque: "); 
          client.print("<b><font color=#E60000>");
          client.print(percentagem, 1);
          client.println("%");
          client.print("</font></b>");
          
          client.print("<p>Quantidade de agua no tanque: ");
          client.print("<b><font color=#E60000>");
          client.print(litros, 1);
          client.println(" litros");
          client.print("</font></b>");
            }
       else {
          client.print("<p>Nivel do Tanque: ");
          client.print("<b>");
          client.print(percentagem, 1);
          client.println("%");
          client.print("</b>");
       
          client.print("<p>Quantidade de agua no tanque: ");
          client.print("<b>");
          client.print(litros, 1);
          client.println(" litros");
          client.print("</b>");
       }
          client.print("</p></body></html>");
          
//FIM DO CÓDIGO HTML A IMPRIMIR NO BROWSER

          file.close();

        } 
        else {
          
//TUDO O QUE NÃO ESTEJA INSERIDO NAS DEFINIÇÕES, É UM ERRO 404

          client.println("HTTP/1.1 404 Not Found");
          client.println("Content-Type: text/html");
          client.println();
          client.println("<h2>Error 404</h2>");
          client.println("");
        }
        break;
      }
    }
    digitalWrite(greenLEDandBEEP, HIGH);
    delay(1);
    digitalWrite(greenLEDandBEEP, LOW);
    delay(1);
    client.stop();
  }
}

//FIM DO PROGRAMA

For the animation you will need to add some javascript to the web page to change the size of the colored-in part of the tank. You could draw the tank with fancy SVG graphics using the Raphael.js graphics library. You could kludge up something simpler like changing the height of an HTML DIV element to match the water level. For that you might use the jQuery javascript library. Or you might find a third-party control library that has a tank gauge you like. Any way you cut it, it's javascript work.

For the refresh, the javascript running in your web page needs to make a periodic AJAX request to the Arduino for the updated data. jQuery has a nice ajax interface to make such requests with; you'll need to add code to your arduino sketch to handle the ajax calls and return just the data instead of the whole page.

-br

Thanks for the answer. I’ve just self-learning arduino and never had a touch with Java/jQuery/AJAX. I’ve been reading some info about how usefull this languages are to almost real time refresh without refreshing the whole page. The thing is that i need an input or some example related to this project that i’m working for me to understand (and see) what are you talking about. If you could please “show me the way”, i will be very thanked. I began learning arduino and a little html this month. :blush:

This is an simple example with java that probably can work in my project, but i need to know how to change the seconds with the ultrasonic sensor values. This exemple increases the height of water.gif every second.

<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<script>
var myVar=setInterval(function(){myTimer()},1000);
var dynamic_height = 5;
function myTimer()
{
	var theImg = document.getElementById("water");
	var d = new Date();
	dynamic_height = 5 + (d.getSeconds()*3);
	theImg.height = dynamic_height;
}
function myStopFunction()
{
clearInterval(myVar);
}
</script>

<body>
<center>
<b>THIS IS THE TITLE</b>
<div style="position:relative; width:360; height:336">
<div style="position:absolute; top:0; left:0"><center><img name="tank" id="tank" src="tank.gif" width="360" height="336"></center></div>
<div style="position:absolute; bottom:6; left:22"><center><input type="image" name="water" id="water" src="water.gif" width="236" height="5">
</center></div>
</div>
</body></html>

No frills analog pin data display.

// zoomkat's meta refresh data iframe test page 12/24/12
// use http://192.168.1.102:84 in your brouser for main page
// http://192.168.1.102:84/data static data page
// http://192.168.1.102:84/datastart meta refresh data page
// for use with W5100 based ethernet shields
// set the refresh rate to 0 for fastest update
// use STOP for single data updates

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 192, 168, 1, 102 }; // arduino ip in lan
byte gateway[] = { 192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port
unsigned long int x=0; //set refresh counter to 0
String readString; 

//////////////////////

void setup(){
  Serial.begin(9600);
    // disable SD SPI if memory card in the uSD slot
  pinMode(4,OUTPUT);
  digitalWrite(4,HIGH);

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  server.begin();
  Serial.println("meta refresh data frame test"); // so I can keep track of what is loaded
}

void loop(){
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
         if (readString.length() < 100) {
          readString += c; 
         } 
        //check if HTTP request has ended
        if (c == '\n') {

          //check get atring received
          Serial.println(readString);

          //output HTML data header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println();

          //generate data page
          if(readString.indexOf("data") >0) {  //checks for "data" page
            x=x+1; //page upload counter
            client.print("<HTML><HEAD>");
            //meta-refresh page every 1 seconds if "datastart" page
            if(readString.indexOf("datastart") >0) client.print("<meta http-equiv='refresh' content='1'>"); 
            client.print("<title>Zoomkat's meta-refresh test</title></head><BODY>
");
            client.print("page refresh number ");
            client.print(x); //current refresh count
            client.print("

");
              //output the value of each analog input pin
             for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
             client.print("analog input ");
             client.print(analogChannel);
             client.print(" is ");
             client.print(analogRead(analogChannel));
             client.print("
");
             }
            client.print("</BODY></HTML>");
           }
          //generate main page with iframe
          else
          {
            client.print("<HTML><HEAD><TITLE>Zoomkat's frame refresh test</TITLE></HEAD>");
            client.print("Zoomkat's Arduino frame meta refresh test 12/24/12");
            client.print("

Arduino analog input data frame:
");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/datastart' target='DataBox' title=''yy''>META-REFRESH</a>");
            client.print("&nbsp;&nbsp;&nbsp;&nbsp;<a href='http://192.168.1.102:84/data' target='DataBox' title=''xx''>SINGLE-STOP</a>
");
            client.print("<iframe src='http://192.168.1.102:84/data' width='35%' height='250' name='DataBox'>");
            client.print("</iframe>
</HTML>");
          }
          delay(1);
          //stopping client
          client.stop();
          //clearing string for next read
          readString="";
        }
      }
    }
  }
}

That Javascript (not Java) example is a good start for the animation. Now if you can arrange to set the variable "dynamic_height" using data from the Arduino you'll have all the pieces you need for a solution.

Look for a simple AJAX example that makes a request to a web server for a single piece of data. In your case, you want the water level reading. You need to add code in the Javascript page to make the AJAX request to the Arduino, and then take the return value and apply it to dynamic_height in the animation code. You'll also need to add code to the Arduino sketch to return just the height value.

-br

For a really nice example take a look at this thread from llukkari, Thread http://arduino.cc/forum/index.php/topic,129122.0.html In particular, his demo for the weather station could be a good example for you to look further at. It uses jQuery mobile library, so this example will work on a range of screen resolutions, smart-phones to desktops. I used this initially, but am now moving to using the Bootstrap.js library which has some other niceties.

He has in my opinion really put together some well structured and fancy code. I have used quite a bit what llukkari has done with my project and have learnt a lot, and still learning.

For some information on what Bill is talking about in terms of data exchange such as using json which is very effective and easy to understand, please read what llukkari has written here http://www.cs.helsinki.fi/u/ljlukkar/iot/

This I believe will give you some more insight into having data updated from a web server to your web browser in a neat way.

Using the Webduino library, I have found that so much of the code is there and I can just call the functions. This makes for easier and neater code in your Arduino. On the client side, the same thing with using a MVC (Model View Controller) also helps make for neater javascript (Not Java, that is a different beast).

You should be able to view some of my code by going to my site and pulling it apart, not by using page source, as it's pretty well all done in javascript and injected into the html body section.

Hope this help a little more.

Blastboot, I sent you a pm. I would like to have a bit more time to write you an example, but just on the time poor side at present, with trying to make progress with my own site.

Kind regards, Paul