help reading web site value

If I enter http://api.thingspeak.com/channels/21652/fields/3/last all that returns is a single number...really want to get this number into my esp201 with arduino code..

can someone show me an example of this..have have tried many examples only ones that work are for posting data not reading it.
I have butchered this code together with no luck...thanks for any help/mocking

#include <ESP8266WiFi.h>
extern "C" {
  #include "user_interface.h"


const char* ssid     = "x";
const char* password = "";

const char* host = "api.thingspeak.com";

}

void setup() {
delay(100);
  Serial.begin(115200);

 
  // We start by connecting to a WiFi network

  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  
  WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");  
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  delay(10000);
  Serial.print("connecting to ");
  Serial.println(host);
  
//   Use WiFiClient class to create TCP connections
    WiFiClient client;
    const int httpPort = 80;
    if (!client.connect(host, httpPort)) {
    delay(1000);
    Serial.println("connection failed");
    delay(10);
    return;
  }
 
 String url = "/channels/21652/fields/3/last";
 Serial.print("Requesting URL: ");
  

//   This will send the request to the server
 client.print(String("GET ") + url + " HTTP/1.1\r\n" +
                "Host: " + host + "\r\n" + 
               "Connection: close\r\n\r\n");
  delay(500);
  
  // Read all the lines of the reply from server and print them to Serial
  while(client.available()){
    String line = client.readStringUntil('\r');
    Serial.print(line);
 delay(500);
  }
 delay(10000); 
}
Connecting to x
....
WiFi connected
IP address: 
192.168.35.42
connecting to api.thingspeak.com
Requesting URL: HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Mon, 03 Aug 2015 20:54:27 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "f3def39a50359c67b3990421011fc09e"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 3715f229-a899-4b32-8772-b53bf0d79b0b

4
19.4
0

The code does something. What does it do?

Please read this:-
How to use this forum
Because your post is breaking the rules about posting code.

Ok thanks I read the post..added the "Code" to my code..I also added some delays to my program and it is actually pulling data... I posted my serial output I had no clue it would look like this in a web browser it will just return 19.4 not. I guess now I have a new..more difficult problem to solve to parse this number out.

.

Connecting to x
....
WiFi connected
IP address: 
192.168.35.42
connecting to api.thingspeak.com
Requesting URL: HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Mon, 03 Aug 2015 20:54:27 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "f3def39a50359c67b3990421011fc09e"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 3715f229-a899-4b32-8772-b53bf0d79b0b

4
19.4
0

X-Request-Id: 3715f229-a899-4b32-8772-b53bf0d79b0b

4
19.4

That blank line there is a clue. Nothing before it matters. What is after it might. The blank line is . So, look for in the stream. When you find it, the length of the record is important. Zero means that you have found the end of the header crap, and can start reading data.

Below is what is returned by the bottom client test code. You will need to parse out the 16.4 from what is returned. The below TextFinder application might be of use.

http://playground.arduino.cc/Code/TextFinder

HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Tue, 04 Aug 2015 00:43:08 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "591cc1c9a3b4cbd908178203fc545d5d"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 525e9e28-02da-4be2-ad04-ddfd88591ff0

4
16.4
0
//zoomkat 11-04-13
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test client GET
//for use with W5100 based ethernet shields
//remove SD card if inserted
//data from myIP server captured in readString 

#include <SPI.h>
#include <Ethernet.h>
String readString;

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "api.thingspeak.com"; // myIP server test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600); 
  Serial.println("client readString test 11/04/13"); // so I can keep track of what is loaded
  Serial.println("Send an e in serial monitor to test"); // what to do to test
  Serial.println();
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) //if something in serial buffer
  {
    byte inChar; // sets inChar as a byte
    inChar = Serial.read(); //gets byte from buffer
    if(inChar == 'e') // checks to see byte is an e
    {
      sendGET(); // call sendGET function below when byte is an e
    }
  }  
} 

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET /channels/21652/fields/3/last HTTP/1.1"); //download text
    client.println("Host: api.thingspeak.com");
    client.println("Connection: close");  //close 1.1 persistent connection  
    client.println(); //end of get request
  } 
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    readString += c; //places captured byte in readString
  }

  //Serial.println();
  client.stop(); //stop client
  Serial.println("client disconnected.");
  Serial.println("Data from server captured in readString:");
  Serial.println();
  Serial.print(readString); //prints readString to serial monitor 
  Serial.println();  
  Serial.println();
  Serial.println("End of readString");
  Serial.println("==================");
  Serial.println();
  readString=""; //clear readString variable

}

Still trying to work my way through this don't know if the above mentioned Textfinder would work in my case as I'm using an esp8266/esp201..I'm trying to parse out this data which is my outdoor temperature, as Pauls suggested..using the ....two hours down..getting nowhere fast...would string substring work...thats what I'm wasting my time with reading about now.

Ok.

You are connecting to an HTTP server.

HTTP servers are (by definition) servers that implement the HPP protocol.

The HTTP protocol is described here: https://www.ietf.org/rfc/rfc2616.txt

You can ignore almost all of it :slight_smile: . The only bits that matter to you are:

  • An HTTP response has a response line, several header lines (terminated with a \r), and a blank line, and a body.

  • The body can be anything - any series of bytes. The number of bytes in the body is given by the Content-Length header. If there is no header, then the only way to find out how long the body is is for the server to transmit the data and then close the pipe.

just changed my code above from using ('\r'); to ('\n') and it only reads the body..slowly understanding..thanks for the help

So in the case of this result.. 19.4 is the data I'm after..the 0 means end of stream. what does the 4 mean.

Connecting to x
....
WiFi connected
IP address:
192.168.35.42
connecting to api.thingspeak.com
Requesting URL: HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Mon, 03 Aug 2015 20:54:27 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "f3def39a50359c67b3990421011fc09e"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 3715f229-a899-4b32-8772-b53bf0d79b0b

4
19.4
0

maxfli:
Still trying to work my way through this don't know if the above mentioned Textfinder would work in my case as I'm using an esp8266/esp201..I'm trying to parse out this data which is my outdoor temperature, as Pauls suggested..using the ....two hours down..getting nowhere fast...would string substring work...thats what I'm wasting my time with reading about now.

I just looked at his module in a short youtube video and it appears to be a nice wireless replacement for the w5100 ethernet shield (and the price is good). If you cave control over what thingspeak sends, then put a unique byte marker before and after the value being sent, like !16.4? that can be easily located in what is received. A crude method I've used in the past is to count carriage returns or line feeds.

yes thanks for the info..that's exactly what I'm trying to figure out..the data I want is directly after the second carriage return

If the 4 is consistent in the returned data, you might try something like below to find where to capture the desired data.

ind1 = readString.indexOf("4\r\n");

ok stil working at it I changed my code to...now I'm getting this json back...would this be easier to work with..

Connecting to x
...................
WiFi connected
IP address: 
192.168.35.42
connecting to api.thingspeak.com
connection failed
connecting to api.thingspeak.com
Requesting URL: HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Tue, 04 Aug 2015 05:34:03 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "58c3125cb2faf4a8cb637ccb83834961"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: a116f961-aa51-4877-a2b7-93906efa871f

47
{"created_at":"2015-08-04T05:33:47Z","entry_id":561465,"field3":"13.6"}
0
#include <ESP8266WiFi.h>
extern "C" {
#include "user_interface.h"

const char* ssid     = "x";
const char* password = "";
const char* host = "api.thingspeak.com";
String line2 = " ";
}

void setup() {
  delay(100);
  Serial.begin(115200);
             
 
      // We start by connecting to a WiFi network
  Serial.println();
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);
  
  WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");  
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  delay(1000);
  Serial.print("connecting to ");
  Serial.println(host);
  
      // Use WiFiClient class to create TCP connections
   WiFiClient client;
   const int httpPort = 80;
   if (!client.connect(host, httpPort)) {
   delay(100);
   Serial.println("connection failed");
   delay(10);
   return;
  }
 
 String url = "/channels/21652/fields/3/last.json";
 Serial.print("Requesting URL: ");
  

     //This will send the request to the server
 client.print(String("GET ") + url + " HTTP/1.1\r\n" +
                "Host: " + host + "\r\n" + 
               "Connection: close\r\n\r\n");
  delay(500);
  
  // Read all the lines of the reply from server and print them to Serial
    while(client.available()){
   String line = client.readStringUntil('\r');
   
   Serial.print(line);
   
  
   //Serial.print(ind1)
 delay(2000);
 
  
  }
  
  delay(10000);
 
 
}
 client.print(String("GET ") + url + " HTTP/1.1\r\n" +
                "Host: " + host + "\r\n" + 
               "Connection: close\r\n\r\n");

There really is no sense pissing away resources on the String class.

 client.print("GET ");
 client.print(url);
 client.println(" HTTP/1.1");
 client.print("Host: ");
 client.println(host);
 client.println("Connection: close");
 client.println();

will do (almost) exactly the same thing, in a much easier to read format (the difference being that it WILL get the carriage returns and line feeds in the correct order and quantity of each).

   while(client.available()){
   String line = client.readStringUntil('\r');
   
   Serial.print(line);
   
  
   //Serial.print(ind1)
 delay(2000);
 
  
  }

Someday, you'll come
to appreciate
code that
lines up nicely.

That looks like shit.

The delay() in there is absolutely pointless.

Ok...made suggested changes to code...and it still works..thanks.fixed spacing as well...with my new serial output..The request to thingspeak is changed..what will the easiest way to get field 3 data out of the reply..all I want is the number 18.8..to be stored as a float..I really appreciate all the help..Thanks

#include <ESP8266WiFi.h>
extern "C" {
#include "user_interface.h"
const char* ssid     = "x";
const char* password = "";
const char* host = "api.thingspeak.com";
}

    void setup() {
delay(100);
Serial.begin(115200);
Serial.println();     // We start by connecting to a WiFi network
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
  
WiFi.begin(ssid, password);
  
  while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}

Serial.println("");
Serial.println("WiFi connected");  
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}

    void loop() {
delay(10000);
Serial.print("connecting to ");
Serial.println(host);
  
     
    WiFiClient client; // Use WiFiClient class to create TCP connections
const int httpPort = 80;
if (!client.connect(host, httpPort)) {
delay(1000);
Serial.println("connection failed");
delay(10);
return;
  }
 
String url = "/channels/21652/feeds/last.json";
Serial.print("Requesting URL: ");
client.print("GET ");
client.print(url);
client.println(" HTTP/1.1");
client.print("Host: ");
client.println(host);
client.println("Connection: close");
client.println();
delay(500);
  
  
    while(client.available()){        // Read all the lines of the reply from server and print them to Serial
String line = client.readStringUntil('\r');
Serial.print(line);

}
delay(10000);
}
onnecting to x
...
WiFi connected
IP address: 
192.168.35.42
connecting to api.thingspeak.com
Requesting URL: HTTP/1.1 200 OK
Server: nginx/1.7.5
Date: Tue, 04 Aug 2015 17:28:23 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: close
Vary: Accept-Encoding
Status: 200 OK
X-Frame-Options: ALLOWALL
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, OPTIONS, DELETE, PATCH
Access-Control-Allow-Headers: origin, content-type, X-Requested-With
Access-Control-Max-Age: 1800
ETag: "2f611421f8c057bfbdb1461464fcdb56"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 7747de26-6ace-4e87-8c48-5633b435a5a3

75
{"created_at":"2015-08-04T17:28:03Z","entry_id":562810,"field1":"79.7","field2":"80.0","field3":"18.8","field4":null}
0

My thinking is I need the 95 to 99 digit after the {.....just have to figure out how to do it...

String line = client.readStringUntil('\r');

You have a String object. Use the indexOf() method to see if field3 exists in the string. If it does, find the : that follows the field3 entry. Then find the comma that follows that.

Create a substring starting after the colon, ending before the comma. Trim white space and then get rid of the start and end quotes.

Then, call the toFloat() method.

My thinking is I need the 95 to 99 digit after the {.....just have to figure out how to do it...

Not unless you KNOW that that is a fixed length string.

According to notepad++, the 95th to 98th characters are ":"1.

Spent about 3 hours could not figure it out...I added this to my original and it works...But I don't think it's up to Pauls type standard....or even mine for that matter...Still a work in progress...Think I need a liitle spoon feeding on this one. If I told you what my day job was you would understand. Tried the indexof could not figure it out..will not give up..still working on it.

  while(client.available()){        // Read all the lines of the reply from server and print them to Serial
String line = client.readStringUntil('\r');
Serial.print(line);
readString += line; //makes the string readString
}

     if (readString.length() >0) {
Serial.println(readString); //see what was received
      
field2 = readString.substring(652, 656); //get the first four characters
field3 = readString.substring(668, 672); //get the next four characters
     
Serial.println(field2);  //print to serial monitor to see parsed results
Serial.println(field3);

int n1 = field2.toInt();
int n2 = field3.toInt();