transferring data from Arduino to the web

ssmith73 · April 9, 2018, 7:13pm

Hi All,
I have a temperature controller setup in my workshop - it uses a couple of nano's (temperature sensing & controller) and a couple of mega's (displays), all communicating with each other using either EasyTransfer (I2c) if they are close enough to each other, or nrf24l01 if not. It generates plenty of data that i would like to put up on the web and monitor when i am bored at work, i'd also like to be able to control i/o's over the web - but that's a stretch goal.

What do i need, keeping things simple? I don't have much experience with accessing the web. My home has a static-ip, and I have a wifi-network in the workshop. I guess the ESP8266 would be used, but i don't understand how i would transfer the data currently sitting in an Arduino-nano to this, so that it can provide access to the web, or how to access the web, or indeed, from the web bacl to the nano.

Any advice on what would work, and what i need to read?

Thanks.

ieee488 · April 9, 2018, 10:25pm

You'll want to get this working on your home network before trying to access it from the outside world.

Decide which Arduino will be the web server which will display the information.

Or do you want yet another Arduino for that purpose?
Don't know enough about your set-up.

I would use an Ethernet shield.

system · April 9, 2018, 10:36pm

Whether you use an Ethernet shield or an ESP8266 to do wireless, the process of making a GET request to a server (Hey, server, run this script to use this data) is the same.

You need to develop the scripts on the server that take the data and shred it. Oh, wait, I meant store it somehow, so that you can access it, having a browser make a different request (Hey, server, run this script to show me what you’ve got).

So, what scripting languages do you know? What server software are you running? Apache, MySqQL and php were made to play together. Setting up a server, with Apache, running PHP scripts, saving data in a MySQL database is pretty easy. Hell, even I was able to do it.

ssmith73 · April 9, 2018, 11:23pm

That all sounds nasty, I have a lot to learn. I write C# for a living, but have not gone near web development. I’ll start reading. Thanks.

JMeller · April 10, 2018, 12:03am

PaulS:
Hell, even I was able to do it.

After nearly 10yrs of watching this forum, even I can appreciate the attempt of modesty.

A year or so age, xl97, Robin2, and others sent me down a good path - https://www.w3schools.com/.

The nano is too small to house any webpage code; the esp8266 has enough to hold a few pages.

Below are snippits from a program used for a card reader; the card reader is active only during a predetermined time period.

Note the “function GetTime()”; this allows real time update every second - no need to refresh page.

ESP8266 Server Setup:

void ServerSetup() {

  ESP8266server.on("/", HTTP_GET, handleRoot);
ESP8266server.on("/ESP8266Time.txt", HTTP_GET, handleGetTime);
}

handleRoot: (html “style” at end of post)

void handleRoot() {                         // When URI / is requested, send a web page with a button to toggle the LED
  
    static unsigned long last = millis();
    if (millis() - last < 3000) {ESP8266server.handleClient();}


  ESP8266server.send(200, "text/html", 
  "<!DOCTYPE html>"
  "<html>"
  "<title>" + String(WEBTITLE) + "</title>"
  "<body onload = \"PageLoad()\">"
  "<style  type=\"text/css\">"
  + String(style) +
  "</style>"
  "<script>"
"          function PageLoad(){"
"          myTimer = setInterval(GetTime, 1000)"
"          }"
"          function GetTime(){"
"            var xhttp = new XMLHttpRequest();"
"            xhttp.onreadystatechange = function() {"
"              if (this.readyState == 4 && this.status == 200) {"
"                 document.getElementById(\"TimeLine\").innerHTML = this.responseText;"
"              }"
"            };"
"            xhttp.open(\"GET\", \"/GetTimeHIST.txt\", true);"
"            xhttp.send(null);"
"          }"
/*
"          function GetHistory(){"
"            var xhttp = new XMLHttpRequest();"
"            xhttp.onreadystatechange = function() {"
"              if (this.readyState == 4 && this.status == 200) {"
"                 document.getElementById(\"HistLine\").innerHTML =\"Card Reader History: \" + this.responseText;"
"              }"
"            };"
"            xhttp.open(\"GET\", \"/GetHIST.txt\", true);"
"            xhttp.send(null);"
"          }"
*/
"          function SetTime(){"
"            document.getElementById(\"TimeLine\").innerHTML =\"Contacting NTP Server... \";"
"            var xhttp = new XMLHttpRequest();"
"            xhttp.onreadystatechange = function() {"
"              if (this.readyState == 4 && this.status == 200) {"

"              }"
"            };"
"            xhttp.open(\"GET\", \"/SetTime.txt\", true);"
"            xhttp.send(null);"
"          }"
  "</script>"
"  <a href=\"http://www.exportmoose.com/\" target=\"_blank\"><img src=\"http://www.exportmoose.com/images/Moose/MainFratMooseLogo.gif\" alt=\"No Internet\"></a>"
//"  <a href=\"http://www.exportmoose.com/\" target=\"_blank\"><img src=\"http://www.exportmoose.com/images/Moose/MainFratMooseLogo.gif\" alt=\"No Internet\">"
//"    </a>"
  "<form action=\"/loginSubmit\" method=\"POST\">"
  "<div class = tbox>"
  "<input type=\"text\" name=\"username\" placeholder=\"Username\">
"
  "</p>"
  "<input type=\"password\" name=\"password\" placeholder=\"Password\">
"
  "</div>"      
  "<div class = btn2>"
  "<input type=\"submit\" value=\"Login\"></form>"
  "</div>"
  "<input type=\"button\" id=\"SetT\" name=\"Time\" value=\"Update Time\" onclick=\"SetTime()\" >"
  "<div class = msg>"
  "<p2 id=\"TimeLine\"></p2>"
  "</div>"

  "<p></p>"
  "<p></p>"
  "<p></p>"
  "<p></p>"
  "<p3 id=\"HistLine\"></p3>"
  "</body>"
  "</html>");
}

handleGetTime:

void handleGetTime(){


LoadDATE();  // retrieve time from NTP server
  if (proALIVE == true){ // if ProMini is communicating, do this..
      ESP8266server.send(200, "text/plain", "<p>" + String(REV) + "</p>" + "Arduino Time:" + String(DATE) + '\t' + "ProMini: ONLINE");
  }else{
      ESP8266server.send(200, "text/plain", "<p>" + String(REV) + "</p>" + "Arduino Time:" + String(DATE) + '\t' + "ProMini: OFFLINE");

  }


}

html style:

const char* style =
"*{"
"  box-sizing: border-box; /* resets box model to avoid dimensional issues*/" 
"  margin: 0;   /* Resets user agent default values */" 
"  padding: 0; /* Resets user agent default values */" 
"   position: relative;" 
"}" 

"h3 {text-shadow: 0 0 5px #FF00FF, 0 0 4px #00AFFF;}" 

"body{background-color:lightblue;}" 

".instr {"
"width: 400px;" 
"font-size: 11px;" 
"font-style: italic;" 
"font-family:Lao UI;" 

"align:left;" 
"line-height: 1.5em;" 
"word-spacing: 0.05em;" 
"letter-spacing: 0.05em;" 
"margin: 1px 10px 1px 10px ;" 
"padding: 5px 0px 15px 0px ;" 
"}" 

".ipaddr {" 
"color: darkred;" 
"margin: 0px 0px 0px 450px;" 
"}" 

".tbox {" 
"margin: 2px 0px 1px 0px ;" 
"padding: 5px 2px 5px 2px ;" 

"width: 174px;" 
"text-color:blue;" 
"/*box-shadow: 2px 2px grey; */" 
"display: inline;" 
"postion: absolute;" 
"}" 

".btn1 {" 
"margin: 1px 10px 1px 10px ;" 
"padding: 2px 4px 2px 2px ;" 
"width: 151px;" 
"/*box-shadow: 2px 2px grey; */" 
"}" 

".btn2 {" 
"align:center;" 
"margin: 1px 0px 1px 10px ;" 
"padding: 2px 0px 3px 3px ;" 
"width: 40px;" 
"/* box-shadow: 2px 2px grey; */" 
"display: inline;" 
"}" 

".btn1:hover, .btn2:hover {" 
"  background: #000FFF;" 
"}" 

".radios{" 
"margin: 10px;" 
"padding: 1px 4px 1px 2px ;" 
"width: 250px;" 
"}" 


".msg {" 
"color:darkblue;" 
"font-size: 18px;" 
"font-family: Lao UI;" 
"font-style: bold;" 
"}" 

".HostSlave{" 
"display: inline;" 
"postion: absolute;"
"}" 

".PassUser{"
"margin: 40px 0px 1px 10px ;" 
"padding: 5px 2px 5px 2px ;" 
"width: 174px;"
"text-color:blue;" 
"}";

Hope this helps to get you started.

Juraj · April 10, 2018, 4:38am

read about Blynk. is not a web page solution, but it is simple to use

Nick_Pyner · April 10, 2018, 4:56am

ssmith73:
Hi All,
I have a temperature controller setup in my workshop -..........it generates plenty of data that i would like to put up on the web and monitor when i am bored at work, i'd also like to be able to control i/o's over the web - but that's a stretch goal.

What do i need, keeping things simple?.........but i don't understand how i would transfer the data currently sitting in an Arduino-nano to this, so that it can provide access to the web, or how to access the web,

This sounds like an exercise for the Internet of Things, and that approach isn't nasty at at all. It is simply a matter of just a few lines of lines of code packaging a datafeed and sending it off to a to an outside service. This involves no Apache, no PHP, no MySQL (whatever the hell they are), and I submit that getting it all to work on your home network first would be an utterly futile exercise.

The essential Arduino line is something like
ret = cosmclient.put(feed, cosmKey);
All the dirty work is done at the other end, and you get to see nice little graphs on your phone in the lunchbreak. The dirty work includes timestamping.

I used Xively for this. This was because it had updates at ten second intervals and, at the time, was free. There was two way traffic, but I only used it to download stored data, and I don't recall what else was on offer - if anything. There is a raft of these facilities, I believe the post popular is Thingspeak. Another is Blynk. All the relevant ones will have Arduino-specific examples. Blynk has a phone app. Others are accessed through a web browser in the normal manner.

The service will involve an extra library or two but your display Mega is the obvious choice, as it is already a central point for data, and should be able to handle that. I use an Ethernet shield. If you want WiFi using ESP8266, it may be possible to have a NODE-MCU ESP8266 replace a Mega.

rw950431 · April 10, 2018, 10:53am

Concur with Nick that online is the way to go. But you will find that Xively has gone very commercial these days so you may wish to check out thingspeak as a free alternative.

ssmith73 · April 11, 2018, 8:29am

Brilliant guys - thanks for your time - IOT would seem to be the way to go. I have a workshop that i have spend far too long with electronics and not enough with wood - i'll be divorced!

Thanks everyone.

ieee488 · April 11, 2018, 1:55pm

Nick_Pyner:
This sounds like an exercise for the Internet of Things, and that approach isn't nasty at at all. It is simply a matter of just a few lines of lines of code packaging a datafeed and sending it off to a to an outside service. This involves no Apache, no PHP, no MySQL (whatever the hell they are), and I submit that getting it all to work on your home network first would be an utterly futile exercise.

The disadvantage of using something like Xively (now owned by Google) or ThingSpeak (now owned by the company that makes Matlab) is that one is at the mercy of that service.

They can start wanting $ for what they used to provide for free.

I suppose since he claims he has lots of time on his hands then that consideration is not relevant.

Nick_Pyner · April 11, 2018, 2:30pm

Thingspeak already has services that they charge for. Nobody can deny that the free services they also have may be charged for at some unspecified, and by no means certain, time in the future, but that is a pretty dumb reason for not using it now.

ieee488 · April 11, 2018, 2:32pm

Nick_Pyner:
Thingspeak already has services that they charge for. Nobody can deny that the free services they also have may be charged for at some unspecified, and by no means certain, time in the future, but that is a pretty dumb reason for not using it now.

I don't think it's "dumb" to not to want to have to do something twice.
It is called considering the pros and cons.

In addition, I think of setting up my own server as a learning experience.
It may not go right, but if I learn something than it is a win situation.

.

Nick_Pyner · April 11, 2018, 4:33pm

ieee488:
It is called considering the pros and cons.

Indeed, and it looks like the OP has done exactly that. Good luck with your server.

Nick_Pyner · April 11, 2018, 4:49pm

Further to ThingSpeak, I see that, if you use their calculator for determining the license category, and choose one device with an update message every 15 seconds, they recommend you take up their free license. The message can carry eight values, so that seems pretty comprehensive.

And not too much of a stretch either. They also have a forum on MIT Android for the other end, including control commands.

ieee488 · April 11, 2018, 7:58pm

Nick_Pyner:
Indeed, and it looks like the OP has done exactly that. Good luck with your server.

I don't think he's done anything.

Brilliant guys - thanks for your time - IOT would seem to be the way to go. I have a workshop that i have spend far too long with electronics and not enough with wood - i'll be divorced!

Hardly a commitment; more like lawyer-speak for I will look into it.

.

ssmith73 · April 12, 2018, 11:23pm

No commitment. Looking at all options, time is not a factor, more a side project. Controller is working well, but it’s a bit remote. The idea of networking the data is mainly just to see how it’s done, and to allow me to keep an eye on things when away from the workshop.

Nick_Pyner · April 12, 2018, 11:47pm

ssmith73:
The idea of networking the data is mainly just to see how it’s done,

While this is all about using the web, you may find using SMS and a phone shield entirely practical. The software effort is about the same. The big advantage of Xively et al is the (more or less) live graphical repesentation, and I have never used my phone shield.

system · May 6, 2021, 11:52am