Yun Web Server/Client Communication Questions

I need some help with a Yun demonstration project I am doing. I am using the yun to control a three color LED and measure/report the current thru the LED device.

I have done a similar project with the Uno and an ETHERNET shield. I saw the code get large quickly due to needed have the HTML code stored as string data in the Arduino code. The Yun with its Linux web server allows the HTML code to be kept seperate from Arduino code.

I have been using examples from the Arduino forum, Arduino Scuola and other web sites. However, I have run into a problem with displaying the measured current value on the client web page. I can not get the
values to be displayed in the desired place on web page.

Below is the HTML code that I am using:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <title>JustGreg's Basic Web Page</title>
    <script type="text/javascript" src="zepto.min.js"></script>
      <script type="text/javascript">
	  function ledon()
	   {
	   $('#content').load('/arduino/ledon');
	   //("Command to turn on LED");
	   }
	  function ledoff()
	   {
       $('#content').load('/arduino/ledoff');
       //("Command to turn off LED");
	   }
	  function selectColor()
	   {
		var pickedColor = color.options[color.selectedIndex];
		var passColor = "/arduino/" + pickedColor.value;
		$('#content').load(passColor);
	   }
    </script>
  </head>
  
  <body onload="setInterval(refresh, 2000);">
    <h1 Align=Center>JustGreg's Arduino Test Web Server
using Zepto(Jquery) and Java Script</h1>
    <h2 Align=Center>This Manlipulates LED devices attached to a Yun</h2>
    <p Align=Center>Press a button to do an operation</p>
    <p>Simple buttons for control of a single LED</p>
    <p Align=Center>
    <span id="content"><!-- 0 --></span>
      <button onclick="ledon()">LED ON</button> 
      <button onclick="ledoff()">LED OFF</button>
    </p><hr>
    <p>Menu Selection type button</p>
    <p Align=Center>
		<select id="color" name="Select Color" size="1" onclick="selectColor()">
		<option value="black">Black</option>
		<option value="red">Red</option>
		<option value="green">Green</option>
		<option value="blue">Blue</option>
		<option value="yellow">Yellow</option>
		<option value="cyan">Cyan</option>
		<option value="magenta">Magenta</option>
		<option value="white">White</option>
		</select>
	</p><HR>
	<p>Now to get information from a sensor

	   <table>
	    <tr>
	    <th>The current in milliamperes is </th>
	    <td id="current">0</td>
	    </tr>
	   </table>	
	</p>
  </body>
</html>

Here is the Arduino code that I am using:

/*
 webRGB 26 Jan 2015 derived from 
 A copy of the webRGB.html and a copy of zepto.js, a  minimized 
 version of jQuery need to be placed in /mnt/sda1/arduino/www.  If the 
 IDE sees the Yun then it is possible to do this with IDE when the files
 are in sketch directory (see original webLed).  To use, open browser to
 http://your_yun_name.local/sd/webRGB.html

 RGB LED pins are:
 3 = redPin, 4 = greenPin, 5 = bluePin for LED in array
 
 The common cathode is thru 100 ohm resistor to GND.  This measures current.
 The resistor for the red pin is 330 ohms. Green resistor is 510 ohms.The blue
 resistor is 270 ohms.  The different resistor values compensate for the volt drops
 of color LED devices and help to provide the right color balance.  

 */

#include <Bridge.h>
#include <YunServer.h>
#include <YunClient.h>

int ledPins[] = {3, 4, 5}; //Red, Green, Blue
const int redPin = 3;
/*LED is common cathode, need +5 for LED ON
  Common cathode is the longest lead, second in from case flat
  for device supplied with 16MHz kit.
  common anode, ON = LOW
*/

const boolean ON = HIGH;
const boolean OFF = LOW;
//array defining LED state for color, these can not be constants.
boolean RED[] = {ON, OFF, OFF};
boolean GREEN[] = {OFF, ON, OFF};
boolean BLUE[] = {OFF, OFF, ON};
boolean YELLOW[] = {ON, ON, OFF};
boolean CYAN[] = {OFF, ON, ON};
boolean MAGENTA[] = {ON, OFF, ON};
boolean WHITE[] = {ON, ON, ON};
boolean BLACK[] = {OFF, OFF, OFF};

// Listen on default port 5555, the webserver on the Yun
// will forward there all the HTTP requests for us.
YunServer server;
String readString;
String command;
String currentVal;

void setup() {
  pinMode(ledPins[0], OUTPUT); //pin 3 for red
  pinMode(ledPins[1], OUTPUT); //pin 4 for green
  pinMode(ledPins[2], OUTPUT); //pinn 5 for blue
  // Bridge startup
  Bridge.begin();

  /* Listen for incoming connection only from localhost
  (no one from the external network could connect) */
  server.listenOnLocalhost();
  server.begin();
}

void loop() {
  // Get clients coming from server
  YunClient client = server.accept();

  // There is a new client?
  if (client) {
    // read the command
    String command = client.readString();
    command.trim(); //kill whitespace
    
  //Process command for LedOn or LedOff
      if (command == "ledon") {
       digitalWrite(redPin, HIGH);
    }
    else if (command == "ledoff") {
       digitalWrite(redPin, LOW);
    }
 
   //Now to process the command from string to the boolean color array
   
    if (command == "black")
     {
      setColor(ledPins, BLACK);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
    if (command == "red")
     {
      setColor(ledPins, RED);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
    if (command == "green")
     {
      setColor(ledPins, GREEN);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
    if (command == "blue")
     {
      setColor(ledPins, BLUE);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
     if (command == "yellow")
     {
      setColor(ledPins, YELLOW);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
    if (command == "cyan")
     {
      setColor(ledPins, CYAN);
      currentVal = currentMeasure();
      client.print(currentVal);
     }
    if (command == "magenta")
     {
      setColor(ledPins, MAGENTA);
      currentVal = currentMeasure();
      client.print(currentVal);;
     }
    if (command == "white")
     {
     setColor(ledPins, WHITE);
     currentVal = currentMeasure();
      client.print(currentVal);
     }
     
    // Close connection and free resources.
    client.stop();
  }

  delay(50); // Poll every 50ms
}

void setColor(int* led, boolean* color)
 {
 for(int i = 0; i < 3; i++){
  digitalWrite(led[i], color[i]);
  }
 }
 
String currentMeasure()
  {
    int rawVal = analogRead(A0);
    float milliAmp = rawVal / 20.46;
    String tmp = String(milliAmp);
    return tmp;
  }

Part of my problem is the examples and other reference books I have been using only have a single control and display element in the HTML. In this case, I am using multiple elements. I want to display the measured current using a simple one row table. I have id attribute (current) for the table data cell.

The examples I have seen have a client.print with HTML code to display the sensor value or just print a string containing the value. I have not been able to make this work.

I think the solution is having client.print use the correct HTML that uses the id attribute for and displays the information in the table cell. The other solution is to get the information from the client.print
into a javascript variable and use .getElementsbyName function to set table cell value. The javascript probably needs to run when the page is "refreshed". However, all my effort at this time have only shown my lack of knowledge of HTML and Javascript coding.

Thank in advance for any help on this either by pointing to example or providing code for HTML or Javascript.

Last question, is there a way to donate some money for the operation of the formun? The forum has been very valuable to me in learning about Arduino.

Maybe this isn't the direction you want to go, but what in my opinion is a more elegant solution is:

  • completely disconnect the sketch from any web stuff. Just let the sketch publish the values in the bridge.
  • on the Linux side start a (python) web framework, and read the values from the bridge in there.

I can type up a basic example if you want to...

Thank you, NewLine for the reply. It does sound interesting and possible solution. Are you suggesting to use Bridge.put(key, value) [in my case, current, string with value] to make the data available for the Linux processor? If you are, then I think the data should also be able with the web server with the URL, my_yun_name.local/data/. I should be able to get the data with javascript. I am not sure about this. But, the Yun guide has this:

"/data" is used to access to the internal key/value storage. The available calls are:

/put/KEY/VALUE : stores a value inside the storage
/get/KEY : obtains the value of the requested key in JSON
/get : obtains the entire storage list in JSON.
/delete : deletes the internal storage

If this is true, then I may have a solution. I just have to learn about using JSON with javascript.

How does this sound? Once again, I am open to suggestions. Thank you in advance for any help.

JustGreg:
Thank you, NewLine for the reply. It does sound interesting and possible solution. Are you suggesting to use Bridge.put(key, value) [in my case, current, string with value] to make the data available for the Linux processor? If you are, then I think the data should also be able with the web server with the URL, my_yun_name.local/data/. I should be able to get the data with javascript. I am not sure about this. But, the Yun guide has this:
If this is true, then I may have a solution. I just have to learn about using JSON with javascript.

How does this sound? Once again, I am open to suggestions. Thank you in advance for any help.

JustGreg,
I was going to suggest similar. JSON libraries are widely available in many languages; tutorials as well.
http://json.org/

Jesse

What you wrote would be half of what I was thinking of. But probably works fine.

I wrote down example code that sends time and a random number from the sketch to the Bridge.
On the linux side I start a tiny webserver that gets these values and publishes them. It does not rely on the existing web stuff already on the Yun. The (small) downside is that you need to do a little bit of stuff on the Linux side.

The final result is:

The example code can be found at Yafa! How to make a fermentation controller using an Arduino Yun.: Publishing sketch values on a web page hosted by the Yun

Thank you to the both of you, NewLine and Jesse. I will try both of your approaches. I am sure I will be busy for awhile. Thanks again!

JustGreg:
Are you suggesting to use Bridge.put(key, value) [in my case, current, string with value] to make the data available for the Linux processor? If you are, then I think the data should also be able with the web server with the URL, my_yun_name.local/data/. I should be able to get the data with javascript.

Yes, I think this would be a good solution. Put the data to the bridge. Then the Linux side can access it with get calls, or even better, like you say, JavaScript in your page can get it directly. For example, the call http://arduino.local/data/get will fetch all of the data at once, returning a json string like this:

{"value":{"mode":"Network Only","sketch":"BarcodeNet - Jan 23 2015 13:02:33","3":"","2":""},"response":"get"}

Here you see I have four keys stored: mode, sketch, 3, and 2, with the last two having empty values.

You can also access individual keys with a call like http://arduino.local/data/get/mode which in this case will return

{"value":"Network Only","key":"mode","response":"get"}

You could also use that mechanism to send data to your sketch with a call like http://arduino.local/data/put/key/value and your sketch can then call Bridge.get() to read the value. Note, however, that unlike using the REST API, this isn't passing messages, it's just overwriting the last value,so if you put the same value twice in a row, the sketch won't see the change. However, you don't need to write nearly as much code for get() as you would for a REST API.

Thanks ShapeShifter for the help.

I made this change to to the function that measures the current to use Bridge.put

void currentMeasure()
  {
    int rawVal = analogRead(A0);
    float milliAmp = rawVal / 20.46;
    String tmp = String(milliAmp);
    tmp= "Current is " + tmp + " mA";
    //Note: Bridge.put does not handle float variables
    Bridge.put("currentVal",tmp);
}

Yes, checking the web page myYunName.local/data/get/currentVal shows this response:

{"value":"Current is 4.89 mA","key":"currentVal","response":"get"}

Which appears to me to be JSON object.

The information is there. I have tried using Javascript ajax call to get the information. But, it was not successful. The library yepto.min.js does have a call $ajax(). Here is the code that I tried:

$.ajax(
		 {
          type: 'GET',
          url: '/data/get/currentVal',
           // data to be added to query string:
           //data: { name: 'Zepto.js' },
           // type of data we are expecting in return:
          dataType: 'json',
          timeout: 300,
          //context: $('body'),
          success: function(data)
          {
           // Supposing this JSON payload was received:
           //   {"project": {"id": 42, "html": "<div>..." }}
           // append the HTML to context object.
           alert(data) //this.append(data.project.html)
          },
          error: function(xhr, type)
          {
          alert('Ajax error!')
          }
          }
         )

I only get the error message, Ajax error. Any suggestions on how to get the JSON object of the /data/Get/currentVal into a javascript variable?

JustGreg:
Yes, checking the web page myYunName.local/data/get/currentVal shows this response: Which appears to me to be JSON object.

Great! You've got that half of the equation working. Yes it's a JSON object.

Unfortunately, trying to fetch that data from javascript and display it on a page is outside of my bailiwick. Someone else will have to step up to the plate to take over on that part.

Good luck!

JustGreg:
Thanks ShapeShifter for the help.

::::SNIP::::

Yes, checking the web page myYunName.local/data/get/currentVal shows this response: Which appears to me to be JSON object.

::::SNIP::::

I only get the error message, Ajax error. Any suggestions on how to get the JSON object of the /data/Get/currentVal into a javascript variable?

JustGreg,

From what I can see you are using a Jquery library to make that AJAX call. Here is your reference.

http://api.jquery.com/jquery.ajax/

Scroll down to the reference on error and you can see how to get the textStatus for the error
Ask questions, if you need help.

Jesse

Thank you, Jesse for the web site pointer. I am using a version of JQuery called yepto. From its description, it has the same API interface as JQuery. The only problem I found on the web was that yepto is not as fast as JQuery. JQuery examples usually point to a Google web site to down load the library code. The library yepto is stand alone, which is why I am trying to use it.

I did have some luck with javascript code to change the line where I want the measured value to show. I used the javascript method document.getElementById(name of id string).innerHTML = string or object to show. It does work well. Now to solve the problem of getting JSON object fro /data/get.

I have been doing a lot of searching for how to get the JSON object. Some examples have half dozen lines of javascript (did not work) or alot of XML calls (did not work).

If I do find the proper code, then this method of using /arduino of the web server to place commands for the Arduino side and using /data to get data back to the server for display will be a good way of making something for the Internet of Things.

I was surprised to find out the Bridge.put() method does not handle float values.

JustGreg:
I was surprised to find out the Bridge.put() method does not handle float values.

Are you saying it handles integers? I thought it only handled strings.

Of course, you can put whatever you want in a string, like "1.23" you just have to parse/convert it on the receiving side.

JustGreg:
Thank you, Jesse for the web site pointer. I am using a version of JQuery called yepto. From its description, it has the same API interface as JQuery. The only problem I found on the web was that yepto is not as fast as JQuery. JQuery examples usually point to a Google web site to down load the library code. The library yepto is stand alone, which is why I am trying to use it.

I did have some luck with javascript code to change the line where I want the measured value to show. I used the javascript method document.getElementById(name of id string).innerHTML = string or object to show. It does work well. Now to solve the problem of getting JSON object fro /data/get.

I have been doing a lot of searching for how to get the JSON object. Some examples have half dozen lines of javascript (did not work) or alot of XML calls (did not work).

If I do find the proper code, then this method of using /arduino of the web server to place commands for the Arduino side and using /data to get data back to the server for display will be a good way of making something for the Internet of Things.

I was surprised to find out the Bridge.put() method does not handle float values.

JustGreg,
a few things

  • You can use Jquery as a standalone.
  • Even with yepto, I would venture to say it is using Jquery as a base or it's a clone. either way it's likely you can use $('#elementID').text("value");.
  • The problem you see with XML is broad; and when I went to tackle this issue, it almost appeared as a conspiracy. There is a JSON solution.
  • Study JQuery a bit and you will find your answer for getting back JSON. With JQuery (and I bet yepto also), it automatically detects JSON or XML and behaves appropriately. Once this works, your biggest problem will be name/value pairs and more than one variable.

Next, Google: jquery detect JSON XML

and you'll find stuff like this:

jQuery- Parse JSON, XML and HTML | jQuery By Example

Lastly, I'm not a huge fan of JQuery. I think it's bloated and not modular enough. Personally, I could write an replacement. None the less, it is an industry standard; understand it and it will reward you.

Get back to us with your results :slight_smile:
Jesse

IMHO going through JSON, etc... feels more complicated than it needs to be.

E.g. the webpage I showed above is made from only this code:

@route('/status')
def status():
    try:
        t =  bc.get("TIME")
        r =  bc.get("RAND")
        return "Time="+t+"
 Rand="+r
    except:
        return "Oops\n"

It just gets the values from the bridge and directly uses those variables in the html returned. (not even real html currently)

NewLine:
(not even real html currently)

And therein lies the rub: that method works fine for returning simple results, but it doesn't scale all that nicely. As the pages and site gets more complex, this technique gets more cumbersome.

12 years ago I wrote a fairly dynamic website where everything was controlled by PHP scripts that loaded html template files, substituted token values with dynamic values, and built more complex pages by combining various partial template files into a single file (depending on the data driving the site) and serving up the combined/updated page. It worked, but it quickly got difficult to manage.

The underlying web technologies have grown a lot since then. I would strongly consider using query and JSON if I were to do it again, even though I understand the "old" way and would have to do a lot of learning to use the "new" way.

Interesting, I know nothing about JQuery, etc...

Just to get some keywords that I can google, what would be the server "thing" on the Yun that could run the web page? (assuming I want to start from scratch and not rely on things the Yun developers already did).

Thank you all, Jesse, NewLine and ShapeShifter for the information and help. I have made some progress, no real success. Here is the code (XML, I think) that gives the best information on the HTTP GET request:

function GetSensor()
	   {
	   var xmlhttp = new XMLHttpRequest();
       var url = "gregyun.local/data/get/";
              
       xmlhttp.onreadystatechange = function() {
       if (xmlhttp.readyState == 1) {$('#gotData').text("Http request submitted")};
       if (xmlhttp.readyState == 2) {$('#gotData').text("Headers Received")};
       if (xmlhttp.readyState == 3) {$('#gotData').text("Reponse being received")};
       if (xmlhttp.readyState == 4 && xmlhttp.status == 200) 
        {
        //var myArr = JSON.parse(xmlhttp.responseText);
        $('#gotData').text(xmlhttp.responseText);
        }
       }
       xmlhttp.open("GET", url, true);  //both true, false failed
       xmlhttp.send(null);
       }

This code with Mozilla FoxFire developer tools shows the server is handling the GET request. However, the server is returning code 404 (requested item is not found). This is telling me that I do not know what is the correct name of the data. I have also dropped yepto for the official Jquery downloaded from Google. It adds about a second to the start up.

At this point, I do not know what to do with the HTML(Jquery/XML) code. Any suggestions will be appreciated. I have an idea on possible solution on the Arduino side, but, I really would like to keep the all the HTML code on the Linux Web Server side.

I did take a look at the "bottle" server suggested by NewLine. I still have the problem of getting the sensor information into the HTML document for display. It looks nice for a quick measurement setup. I did find it interesting the referenced web page used PIP to do things. PIP brought back memories of Digital Equipment Corp. (DEC) command language. PIP with DEC PDP11 and VAX equipment stood for Peripheral Interchange Program, which copied files and a lot more. It only shows even the old comes back around again.

JustGreg:

       var url = "gregyun.local/data/get/";

Try taking the trailing slash off of the end of the URL. With that slash, it might be looking for (and not finding) the name of the key you want.

PIP brought back memories of Digital Equipment Corp. (DEC) command language. PIP with DEC PDP11 and VAX equipment stood for Peripheral Interchange Program, which copied files and a lot more. It only shows even the old comes back around again.

Wow, a blast from the past - I remember using PIP for just about everything on PDP-8's and PDP-11's: directories, copying files, making/reading tapes, etc. Those were the days...

NewLine:
Just to get some keywords that I can google,

If you find a good source, let me know. I know just enough about it to know I want to know a lot more about it.

what would be the server "thing" on the Yun that could run the web page? (assuming I want to start from scratch and not rely on things the Yun developers already did).

You'd actually still use a lot of what the Yun developers already did. You'd use Bridge.put() to set the values to be read, and you'd use the built-in web server to serve up pages from the /www folder in the root of the SD card, using a base URL of arduino.local/sd/

The issue is getting the values received from arduino.local/data/get into the objects on the web page.

I got it to work :wink: , sort of. Yes, Shapeshifter, I had the url incorrect! Here is the function code to get the data:

function GetSensor()
	   {
	   var xmlhttp = new XMLHttpRequest();
       var url = "/data/get/";
       xmlhttp.open("GET", url, false); //false is synchronous mode       
       xmlhttp.onreadystatechange = function() {
       if (xmlhttp.readyState == 4 && xmlhttp.status == 200) 
        {
        //var SensorArr = JSON.parse(xmlhttp.responseText);
        $('#gotData').text(xmlhttp.responseText);
        //$('#gotData').text(SensorArr.value);
        }
       }
       xmlhttp.send(null);
       }

Now for the sort of working, the xmlhttp.responseText is not a JSON string. Here is the responseText:

{"value":{"currentVal":"Current is 6.45 mA"},"response":"get"}

The other problem is one does not get the correct current data. It is from the last change of the LED status and not most recent change. Once again, any ideas on how to flush the old data before getting the new will be appreciated. It is nice to have some progress on this. Web servers are both frustrating and interesting.

ShapeShifter, another oldie, but goodie for you, Do you remember paper tapes for boot loaders? I always made an extra copy after the current one ripped from use and one had to use the spare ;D