Pages: [1]   Go Down
Author Topic: Reading a website and selecting imformation among the code return  (Read 371 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Newbie
*
Karma: 0
Posts: 10
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Im making a smart clock project now and i wanna get information from website posting weather information by using the Web client code.However, i can only read "www.google.com" but not "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183" the page which i wanna get information from.
Besides. by looking at the example given in "http://bildr.org/page/11/"
code:
Code:
//ARDUINO 1.0+ ONLY
//ARDUINO 1.0+ ONLY
#include <Ethernet.h>
#include <SPI.h>

////////////////////////////////////////////////////////////////////////
//CONFIGURE
////////////////////////////////////////////////////////////////////////
byte server[] = { 174,123,231,247 }; //ip Address of the server you will connect to

//The location to go to on the server
//make sure to keep HTTP/1.0 at the end, this is telling it what type of file it is
String location = "/~bildr/examples/ethernet/ HTTP/1.0";

// if need to change the MAC address (Very Rare)
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
////////////////////////////////////////////////////////////////////////

EthernetClient client;

char inString[32]; // string for incoming serial data
int stringPos = 0; // string index counter
boolean startRead = false; // is reading?

void setup(){
  Ethernet.begin(mac);
  Serial.begin(9600);
}

void loop(){
  String pageValue = connectAndRead(); //connect to the server and read the output

  Serial.println(pageValue); //print out the findings.

  delay(5000); //wait 5 seconds before connecting again
}

String connectAndRead(){
  //connect to the server

  Serial.println("connecting...");

  //port 80 is typical of a www page
  if (client.connect(server, 80)) {
    Serial.println("connected");
    client.print("GET ");
    client.println(location);
    client.println();

    //Connected - Read the page
    return readPage(); //go and read the output

  }else{
    return "connection failed";
  }

}

String readPage(){
  //read the page, and capture & return everything between '<' and '>'

  stringPos = 0;
  memset( &inString, 0, 32 ); //clear inString memory

  while(true){

    if (client.available()) {
      char c = client.read();

      if (c == '<' ) { //'<' is our begining character
        startRead = true; //Ready to start reading the part
      }else if(startRead){

        if(c != '>'){ //'>' is our ending character
          inString[stringPos] = c;
          stringPos ++;
        }else{
          //got what we need here! We can disconnect now
          startRead = false;
          client.stop();
          client.flush();
          Serial.println("disconnecting.");
          return inString;

        }

      }
    }

  }

}
, i find it possible to select information by recognizing "<" and ">", so my idea is to recognize something like<span class="wx-temp"> and<span> instead of simply recognizing "<"">". But when i tried to implement this method, it returns nothing in serial monitor, i think it is because client.read reads one character at a time.so how shall i change the code? or can anyone of u bring me any better solutions for this problem?
Logged

Massachusetts, USA
Offline Offline
Tesla Member
***
Karma: 208
Posts: 8856
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

You have to add a null terminator to your buffer before you convert it to a String.

You also have to make sure you don't over-fill the buffer.

Code:

String readPage() {
  //read the page, and capture & return everything between '<' and '>'

  stringPos = 0;
  memset( &inString, 0, 32 ); //clear inString memory

  while(true){

    if (client.available()) {
      char c = client.read();

      switch (c) {
       
      case '<':
        stringPos = 0;
        startRead = true; //Ready to start reading the part
        break;

      case '>':
        if (startRead) {
          inString[stringPos++] = '\0'; // Add null terminator
          client.stop();
          client.flush();
          Serial.println("disconnecting.");
          return inString;
        }

      default:
        if (startRead && stringPos < (sizeof instring - 1)) {
          inString[stringPos++] = c;
        }
      }
    }
  }
}
Logged

Send Bitcoin tips to: 1L3CTDoTgrXNA5WyF77uWqt4gUdye9mezN
Send Litecoin tips to : LVtpaq6JgJAZwvnVq3ftVeHafWkcpmuR1e

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 615
Posts: 49388
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
However, i can only read "www.google.com" but not "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183" the page which i wanna get information from.
Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 10
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.

when i read the first one...it returns Html codes...but when i read the second one...there is nothing but "connecting...."
Logged

Offline Offline
Newbie
*
Karma: 0
Posts: 10
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.
Code:
/*
  DNS and DHCP-based Web client
 
This sketch connects to a website (http://www.google.com)
using an Arduino Wiznet Ethernet shield.
 
Circuit:
* Ethernet shield attached to pins 10, 11, 12, 13
 
created 18 Dec 2009
by David A. Mellis
modified 9 Apr 2012
by Tom Igoe, based on work by Adrian McEwen
 
*/

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0x00, 0x03, 0x0D, 0xE9, 0xAD, 0x27 }; //请自行替换成自己的mac地址
char serverName[] = "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183";

// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
// Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting…");

  // if you get a connection, report back via serial:
 
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /s?wd=arduino HTTP/1.0");
    client.println();
  }
  else {
    // if you didn’t get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server’s disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}
here is the code i got and it only returns:
connecting…
connected

disconnecting.
so it actually connected but could read nothing.
do u know why?
Logged

Seattle, WA USA
Offline Offline
Brattain Member
*****
Karma: 615
Posts: 49388
Seattle, WA USA
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The server that you want to connect to is NOT "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183". It is "www.weather.com". The rest of that is supposed to be part of the GET request.

The connection should result in more than the one character you are reading. At a minimum, you should be reading the reply using a while statement, not an if statement.
Logged

0
Offline Offline
Tesla Member
***
Karma: 145
Posts: 9636
Arduino rocks
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
so it actually connected but could read nothing.
do u know why?

You need to find another source for your weather data. the page you want to capture has way too much useless churn to deal with (look at the html source code on the weather page). use the code below to display in the serial monitor what is downloaded.

Code:
//zoomkat 9-22-12
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "www.weather.com"; // zoomkat's test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600);
  Serial.println("Better client test 9/22/12"); // so I can keep track of what is loaded
  Serial.println("Send an e in serial monitor to test"); // what to do to test
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) //if something in serial buffer
  {
    byte inChar; // sets inChar as a byte
    inChar = Serial.read(); //gets byte from buffer
    if(inChar == 'e') // checks to see byte is an e
    {
      sendGET(); // call sendGET function below when byte is an e
    }
  } 
}

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET /weather/right-now/Enschede+Netherlands+NLXX0183 HTTP/1.0"); //download text
    client.println("Host: www.weather.com");
    client.println(); //end of get request
  }
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    Serial.print(c); //prints byte to serial monitor
  }

  Serial.println();
  Serial.println("disconnecting.");
  Serial.println("==================");
  Serial.println();
  client.stop(); //stop client

}

Logged

Consider the daffodil. And while you're doing that, I'll be over here, looking through your stuff.   smiley-cool

Pages: [1]   Go Up
Jump to: