Go Down

Topic: Reading a website and selecting imformation among the code return (Read 399 times) previous topic - next topic

larrylai

Im making a smart clock project now and i wanna get information from website posting weather information by using the Web client code.However, i can only read "www.google.com" but not "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183" the page which i wanna get information from.
Besides. by looking at the example given in "http://bildr.org/page/11/"
code:
Code: [Select]
//ARDUINO 1.0+ ONLY
//ARDUINO 1.0+ ONLY
#include <Ethernet.h>
#include <SPI.h>

////////////////////////////////////////////////////////////////////////
//CONFIGURE
////////////////////////////////////////////////////////////////////////
byte server[] = { 174,123,231,247 }; //ip Address of the server you will connect to

//The location to go to on the server
//make sure to keep HTTP/1.0 at the end, this is telling it what type of file it is
String location = "/~bildr/examples/ethernet/ HTTP/1.0";

// if need to change the MAC address (Very Rare)
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
////////////////////////////////////////////////////////////////////////

EthernetClient client;

char inString[32]; // string for incoming serial data
int stringPos = 0; // string index counter
boolean startRead = false; // is reading?

void setup(){
  Ethernet.begin(mac);
  Serial.begin(9600);
}

void loop(){
  String pageValue = connectAndRead(); //connect to the server and read the output

  Serial.println(pageValue); //print out the findings.

  delay(5000); //wait 5 seconds before connecting again
}

String connectAndRead(){
  //connect to the server

  Serial.println("connecting...");

  //port 80 is typical of a www page
  if (client.connect(server, 80)) {
    Serial.println("connected");
    client.print("GET ");
    client.println(location);
    client.println();

    //Connected - Read the page
    return readPage(); //go and read the output

  }else{
    return "connection failed";
  }

}

String readPage(){
  //read the page, and capture & return everything between '<' and '>'

  stringPos = 0;
  memset( &inString, 0, 32 ); //clear inString memory

  while(true){

    if (client.available()) {
      char c = client.read();

      if (c == '<' ) { //'<' is our begining character
        startRead = true; //Ready to start reading the part
      }else if(startRead){

        if(c != '>'){ //'>' is our ending character
          inString[stringPos] = c;
          stringPos ++;
        }else{
          //got what we need here! We can disconnect now
          startRead = false;
          client.stop();
          client.flush();
          Serial.println("disconnecting.");
          return inString;

        }

      }
    }

  }

}

, i find it possible to select information by recognizing "<" and ">", so my idea is to recognize something like<span class="wx-temp"> and<span> instead of simply recognizing "<"">". But when i tried to implement this method, it returns nothing in serial monitor, i think it is because client.read reads one character at a time.so how shall i change the code? or can anyone of u bring me any better solutions for this problem?

johnwasser

You have to add a null terminator to your buffer before you convert it to a String.

You also have to make sure you don't over-fill the buffer.

Code: [Select]


String readPage() {
  //read the page, and capture & return everything between '<' and '>'

  stringPos = 0;
  memset( &inString, 0, 32 ); //clear inString memory

  while(true){

    if (client.available()) {
      char c = client.read();

      switch (c) {
       
      case '<':
        stringPos = 0;
        startRead = true; //Ready to start reading the part
        break;

      case '>':
        if (startRead) {
          inString[stringPos++] = '\0'; // Add null terminator
          client.stop();
          client.flush();
          Serial.println("disconnecting.");
          return inString;
        }

      default:
        if (startRead && stringPos < (sizeof instring - 1)) {
          inString[stringPos++] = c;
        }
      }
    }
  }
}
Send Bitcoin tips to: 1L3CTDoTgrXNA5WyF77uWqt4gUdye9mezN
Send Litecoin tips to : LVtpaq6JgJAZwvnVq3ftVeHafWkcpmuR1e

PaulS

Quote
However, i can only read "www.google.com" but not "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183" the page which i wanna get information from.

Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.

larrylai



Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.


when i read the first one...it returns Html codes...but when i read the second one...there is nothing but "connecting...."

larrylai



Why can't you read from the second site? When you try, something is returned to the Arduino. What is it?

Using the String class at all is a really lousy idea.

Code: [Select]
/*
  DNS and DHCP-based Web client
 
This sketch connects to a website (http://www.google.com)
using an Arduino Wiznet Ethernet shield.
 
Circuit:
* Ethernet shield attached to pins 10, 11, 12, 13
 
created 18 Dec 2009
by David A. Mellis
modified 9 Apr 2012
by Tom Igoe, based on work by Adrian McEwen
 
*/

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0x00, 0x03, 0x0D, 0xE9, 0xAD, 0x27 }; //?????????mac??
char serverName[] = "www.weather.com/weather/right-now/Enschede+Netherlands+NLXX0183";

// Initialize the Ethernet client library
// with the IP address and port of the server
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
// Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting…");

  // if you get a connection, report back via serial:
 
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /s?wd=arduino HTTP/1.0");
    client.println();
  }
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}

here is the code i got and it only returns:
connectingâ?¦
connected

disconnecting.
so it actually connected but could read nothing.
do u know why?

Go Up