Getting a webpage with Wiznet ethernet shield

Evening all!

Arduino newbie here. I'm a pretty technical person, so no need to be gentle - however, my networking knowledge is somewhat limited, and that is where this project is falling over.

I've got an UNO talking to a Wiznet ethernet shield. My aim is to 'get' this website: www.bramblemet.co.uk/wap/, then have a look at the numbers and move a needle on a dial so I have an indication of windspeed without actually checking the 'net.

I'm stuck at the first hurdle: I can't get the website. I can make HTTP requests to other sites just fine, but the site above does something funky that I don't understand. My code is simply the ethernet web client example with the ip address changed to point it at the right server.

I've tried various HTTP requests, but they all end in errors being sent back from the server.

/*
  Web client
 
 This sketch connects to a website (http://www.google.com)
 using an Arduino Wiznet Ethernet shield. 
 
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13
 
 created 18 Dec 2009
 by David A. Mellis
 
 */

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 192,168,1,177 };
byte server[] = { 217,174,242,1 }; // Bramblemet server

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
Client client(server, 80);

void setup() {
  // start the Ethernet connection:
  Ethernet.begin(mac, ip);
  // start the serial library:
  Serial.begin(9600);
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect()) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /wap/ HTTP/1.0");
    client.println();
  } 
  else {
    // kf you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    for(;;)
      ;
  }
}

Can someone point me in the right direction please? I understand that the server is using something funky as the address on my browser does funky things - I'm just not sure how to attack this from the Arduino!

I've tried various HTTP requests, but they all end in errors being sent back from the server.

While those errors may not mean anything to you, they might to us.

Post the various GET requests you have tried, and the response you get from each.

It appears that a re-direct is occuring. Perhaps you just need to do a GET on the redirected site, instead.

If you can't use the IP address in a browser and get your page, then it probably won't work with the arduino client. Often big web sites aren't accessable via a direct IP address. Other sites may use techniques that make it difficult to "scrape" the prime data on their site without accessing via a browser.

It works! Sort of...

I used a network packet analyser to compare the HTTP requests from my browser and the Arduino. The Ardunio wasn't making the request quite right. Once I included the host name the data sent back contained the address for the redirect. Copying/pasting that into the code and trying again resulted in the right page being reached.

Sorry, yes, I should have included the errors from the server. (Generally they were 'Bad Request').

Thanks for your help chaps. I'm sure I'll be back soon. :smiley:

Reckless.