Arduino Ethernet and http GET

Projects Networking, Protocols, and Devices
1

Hello all. Been scouring the forum and see many others have had similar problems but I've been unable to make any headway. I'm using the arduino web client example code to access a file on my web server using GET. I cannot make it to work as I only get "404 Not found" or "400 Bad request" errors. Here's my code :

char serverName[] = "www.hareti.co.uk";
 
 if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /test.txt HTTP/1.0\r\n");
    client.println();
  }

which results in :

HTTP/1.1 404 Not Found
Date: Tue, 18 Dec 2012 21:23:25 GMT
Server: Apache/2.2.22 (Unix) mod_ssl/2.2.22 OpenSSL/0.9.8e-fips-rhel5 DAV/2 mod_bwlimited/1.4
Accept-Ranges: bytes
Connection: close
Content-Type: text/html

404 Not Found

etc...

I've also tried :

char serverName[] = "www.hareti.co.uk";
	
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /test.txt HTTP/1.0\r\n");
    client.println("Host: www.hareti.co.uk\r\n");
    client.println();
  }

but still only get a 404 not found error. I've tried changing the to HTTP 1.1 but this results in the following :

HTTP/1.1 400 Bad Request
Date: Tue, 18 Dec 2012 21:34:08 GMT
Server: Apache/2.2.22 (Unix) mod_ssl/2.2.22 OpenSSL/0.9.8e-fips-rhel5 DAV/2 mod_bwlimited/1.4
Content-Length: 376
Connection: close
Content-Type: text/html; charset=iso-8859-1

400 Bad Request

For two days I've been trying various combinations of HTTP versions, urls and "\n\r" with nothing but 404s and 400s. I've even installed a network traffic monitor on my PC to see how a web browser composes a request for the file but even copying this doesn't work. Chrome sends :

GET http://www.hareti.co.uk/test.txt HTTP/1.1
Host: www.hareti.co.uk
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en-US;q=0.8,en;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: __utmz=125565075.1351194686.29.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); PHPSESSID=963121c0a350be6962201519fb4fe1d0; __utma=125565075.1148824640.1316869956.1355782628.1355819117.31; __utmc=125565075
If-None-Match: "6cd00e0-20-4d12595427bc0"
If-Modified-Since: Tue, 18 Dec 2012 19:34:15 GMT

which returns the txt file perfectly. Changing my code to :

  char serverName[] = "www.hareti.co.uk";
if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET http://www.hareti.co.uk/test.txt HTTP/1.1\r\n");
    client.println("Host: www.hareti.co.uk\r\n");
    client.println();
  }

results in :

HTTP/1.1 400 Bad Request
Date: Tue, 18 Dec 2012 22:05:07 GMT
Server: Apache/2.2.22 (Unix) mod_ssl/2.2.22 OpenSSL/0.9.8e-fips-rhel5 DAV/2 mod_bwlimited/1.4
Content-Length: 371
Connection: close
Content-Type: text/html; charset=iso-8859-1

400 Bad Request

I can't figure out what I'm doing wrong. Does anyone have any suggestions?

2

Below is some test code that gets a test web page file that sounds similar to your ssetup.

//zoomkat 9-22-12
//simple client test
//for use with IDE 1.0.1
//with DNS, DHCP, and Host
//open serial monitor and send an e to test
//for use with W5100 based ethernet shields

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address

char serverName[] = "web.comporium.net"; // zoomkat's test web page server
EthernetClient client;

//////////////////////

void setup(){

  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }

  Serial.begin(9600); 
  Serial.println("Better client test 9/22/12"); // so I can keep track of what is loaded
  Serial.println("Send an e in serial monitor to test"); // what to do to test
}

void loop(){
  // check for serial input
  if (Serial.available() > 0) //if something in serial buffer
  {
    byte inChar; // sets inChar as a byte
    inChar = Serial.read(); //gets byte from buffer
    if(inChar == 'e') // checks to see byte is an e
    {
      sendGET(); // call sendGET function below when byte is an e
    }
  }  
} 

//////////////////////////

void sendGET() //client function to send/receive GET request data.
{
  if (client.connect(serverName, 80)) {  //starts client connection, checks for connection
    Serial.println("connected");
    client.println("GET /~shb/arduino.txt HTTP/1.0"); //download text
    client.println("Host: web.comporium.net");
    client.println(); //end of get request
  } 
  else {
    Serial.println("connection failed"); //error message if no client connect
    Serial.println();
  }

  while(client.connected() && !client.available()) delay(1); //waits for data
  while (client.connected() || client.available()) { //connected or data available
    char c = client.read(); //gets byte from ethernet buffer
    Serial.print(c); //prints byte to serial monitor 
  }

  Serial.println();
  Serial.println("disconnecting.");
  Serial.println("==================");
  Serial.println();
  client.stop(); //stop client

}
3 
    // Make a HTTP request:
    client.println("GET /test.txt HTTP/1.0\r\n");
    client.println();

That should work. Perhaps get rid of the \r.

I suppose you actually have test.txt in the root directory of the web server?

4

This should work if the server uses virtual hosting.

    client.println("GET /test.txt HTTP/1.1");
    client.println("Host: www.hareti.co.uk");
    // now send the blank line
    client.println("Connection: close\r\n");

No double cr/lf (blank line) until end of header. You were sending too many too soon.

5

Thanks SurferTim, I was getting carried away with the cr/lf s. Works fine now.

6 
    client.println("GET /test.txt HTTP/1.0\r\n");

Oh, of course. That line does two newlines.

7

Can you look at my post, the error is actually 400 Bad Request, but I dont know what is wrong with the request. How can I find out?

https://forum.arduino.cc/index.php?topic=399805.0