Arduino Ethernet Shield Unable to connect to any website other than google.com

Hello,

I have tested using the WebClient Ethernet example to connect to google.com with my Arduino R3 Ethernet Shield and it works. However, when I attempt to connect to any website other than google.com, it gives the bad request error. Here is my code:

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xCC, 0x7A };
char server[] = "dell.com";    // name address for Dell (using DNS)

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,177);

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET / HTTP/1.1\r\n");
    client.println("Host: thoughtpool.org\r\n");
    client.println("Connection: close\r\n\r\n");
    client.println();
  } 
  else {
    // kf you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}

This code outputs “Error 400: Bad Request” in the serial monitor when run. However, when this page is accessed through a web browser, there is a legitimate website at that URL. I have searched on the internet and on the Arduino forums, but none of the suggestions for similar issues worked (all I know of are implemented in this code).

For google.com and any google.com subdomains, however, this code works perfectly. On all other websites, though, it gives “Bad Request” error.

Any suggestions as to why this may be happening?

Thank you in advance for your help.

Too many "\r\n" here. That signifies end of header, and it isn't finished sending the header.

  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET / HTTP/1.1\r\n");
    client.println("Host: thoughtpool.org\r\n");
    client.println("Connection: close\r\n\r\n");
    client.println();
  }

This is the way it should look. The host must match the server domain name.

  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET / HTTP/1.1");
    client.println("Host: dell.com");
    client.println("Connection: close\r\n");
  }

Thank you for the quick reply. I tried SurferTim’s solution, but unfortunately, it gave a “301 Moved Permanently” error. My updated code is below:

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0x90, 0xA2, 0xDA, 0x0D, 0xCC, 0x7A };
// if you don't want to use DNS (and reduce your sketch size)
// use the numeric IP instead of the name for the server:
//IPAddress server(74,125,232,128);  // numeric IP for Google (no DNS)
char server[] = "dell.com";    // name address for Google (using DNS)

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192,168,0,177);

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    // try to congifure using IP address instead of DHCP:
    Ethernet.begin(mac, ip);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET / HTTP/1.1");
    client.println("Host: dell.com");
    client.println("Connection: close\r\n");
    client.println();
  } 
  else {
    // kf you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}

When I tried connecting to the new address that the page was “moved permanently” to (“http://www.dell.com” as opposed to just “dell.com”), it just connected and then immediately disconnected without even accessing the website.

Are there any other fixes I may need to make to my code?

Thank you again for your help.

It is Dell's server. It does some type of weird redirect that I'm not going to spend time decoding. Try another site.

Simple client test code to see if you can make the connection.

//zoomkat 9-22-12
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test
//for use with W5100 based ethernet shields
//remove SD card if inserted

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "web.comporium.net"; // zoomkat's test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600); 
  Serial.println("Better client test 9/22/12"); // so I can keep track of what is loaded
  Serial.println("Send an e in serial monitor to test"); // what to do to test
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) //if something in serial buffer
  {
    byte inChar; // sets inChar as a byte
    inChar = Serial.read(); //gets byte from buffer
    if(inChar == 'e') // checks to see byte is an e
    {
      sendGET(); // call sendGET function below when byte is an e
    }
  }  
} 

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET /~shb/arduino.txt HTTP/1.0"); //download text
    client.println("Host: web.comporium.net");
    client.println(); //end of get request
  } 
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    Serial.print(c); //prints byte to serial monitor 
  }

  Serial.println();
  Serial.println("disconnecting.");
  Serial.println("==================");
  Serial.println();
  client.stop(); //stop client

}

Thanks for your quick replies.

It turns out that Dell's servers do indeed do redirecting, which was throwing off the Ethernet Shield.

Zoomkat's client test code did work for me. I changed Zoomkat's code to match up with the webpage I am trying to access, and it worked! I am still not sure why Zoomkat's code worked and mine didn't, but that's something I will have to look into myself :).

Thank you again for your timely and informative replies. I really appreciate it.