Using Ethernet Shield to read text from a web page

I recently picked up the Arduino Ethernet shield and I'm getting started on the first piece of my project.

I want to connect to NFL.com | Official Site of the National Football League and return the text from that page. Currently it displays

{"ss":[["Sun","8:00","Pregame",,"ARI",,"NO",,,,"55760",,"PRE0","2012"]]}

Eventually I'll be using the data that is returned, but that part isn't important now.

I got the DNSWebClient Example code working and tried to modify it for my project. I just changed the URL for my needs.

/*
  DNS and DHCP-based Web client 
 
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13 
 */

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
byte mac[] = {  0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02 };
char serverName[] = "www.nfl.com";


String location = "/~liveupdate/scorestrip/scorestrip.json HTTP/1.0";

// Initialize the Ethernet client library
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");
    client.println();
  } 
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}

Here is what is displayed in the serial monitor.

connected
HTTP/1.0 400 Bad Request
Server: AkamaiGHost
Mime-Version: 1.0
Content-Type: text/html
Content-Length: 255
Expires: Sun, 22 Jul 2012 20:18:33 GMT
Date: Sun, 22 Jul 2012 20:18:33 GMT
Connection: close

<HTML><HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD><BODY>
<H1>Invalid URL</H1>
The requested URL "www&#46;nfl&#46;com&#47;liveupdate&#47;scorestrip&#47;scorestrip&#46;json", is invalid.<p>
Reference&#32;&#35;9&#46;1f4d1160&#46;1342988313&#46;0
</BODY></HTML>

disconnecting.

I'm don't fully understand what HTTP request I should be using. I haven't found a tutorial that explains how that code works and I'm not sure what I search for in order to find more information.

Any assistance is greatly appreciated. Thanks.

Read what is being returned by the web server. It is telling you the problem.

The browser that you typed the http string into parsed it into host name and URL. You are expected to be smart enough to do the same thing. The URL is everything after the host name, including the /.

Thanks guys. I got it working
changed

client.println("GET www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");

to

client.println("GET http://www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");

and it returned

connecting...
connected
HTTP/1.0 200 OK
Server: Apache
Last-Modified: Sun, 22 Jul 2012 22:33:50 GMT
ETag: "841a4-48-b9250f80"
Content-Type: text/plain
Cache-Control: max-age=15
Expires: Sun, 22 Jul 2012 22:34:15 GMT
Date: Sun, 22 Jul 2012 22:34:00 GMT
Content-Length: 72
Connection: close
X-Akamai-Edgescape: country_code=US

{"ss":[["Sun","8:00","Pregame",,"ARI",,"NO",,,,"55760",,"PRE0","2012"]]}
disconnecting.

Now I just need to filter the header text, but that will be easy.

Did you get to parse de json?

Greets,
Roberto

would also like to know....

i can use this in a current project where a page returns raw text