Topic: Using Ethernet Shield to read text from a web page (Read 19585 times)

DPax · Jul 22, 2012, 10:44 pm

I recently picked up the Arduino Ethernet shield and I'm getting started on the first piece of my project.

I want to connect to http://www.nfl.com/liveupdate/scorestrip/scorestrip.json and return the text from that page. Currently it displays

Code: [Select]


{"ss":[["Sun","8:00","Pregame",,"ARI",,"NO",,,,"55760",,"PRE0","2012"]]}

Eventually I'll be using the data that is returned, but that part isn't important now.

I got the DNSWebClient Example code working and tried to modify it for my project. I just changed the URL for my needs.

Code: [Select]


/*
  DNS and DHCP-based Web client 
 
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13 
 */

#include <SPI.h>
#include <Ethernet.h>

// Enter a MAC address for your controller below.
byte mac[] = {  0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02 };
char serverName[] = "www.nfl.com";


String location = "/~liveupdate/scorestrip/scorestrip.json HTTP/1.0";

// Initialize the Ethernet client library
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");
    client.println();
  } 
  else {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}

Here is what is displayed in the serial monitor.

Code: [Select]


connected
HTTP/1.0 400 Bad Request
Server: AkamaiGHost
Mime-Version: 1.0
Content-Type: text/html
Content-Length: 255
Expires: Sun, 22 Jul 2012 20:18:33 GMT
Date: Sun, 22 Jul 2012 20:18:33 GMT
Connection: close

<HTML><HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD><BODY>
<H1>Invalid URL</H1>
The requested URL "www&#46;nfl&#46;com&#47;liveupdate&#47;scorestrip&#47;scorestrip&#46;json", is invalid.<p>
Reference&#32;&#35;9&#46;1f4d1160&#46;1342988313&#46;0
</BODY></HTML>

disconnecting.

I'm don't fully understand what HTTP request I should be using. I haven't found a tutorial that explains how that code works and I'm not sure what I search for in order to find more information.

Any assistance is greatly appreciated. Thanks.

cmiyc · Jul 22, 2012, 10:55 pm

Read what is being returned by the web server. It is telling you the problem.

PaulS · Jul 22, 2012, 10:59 pm

The browser that you typed the http string into parsed it into host name and URL. You are expected to be smart enough to do the same thing. The URL is everything after the host name, including the /.

DPax · Jul 23, 2012, 12:38 am

Thanks guys. I got it working
changed

Code: [Select]

client.println("GET www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");
to

Code: [Select]

client.println("GET http://www.nfl.com/liveupdate/scorestrip/scorestrip.json HTTP/1.0");

and it returned

Code: [Select]

connecting...
connected
HTTP/1.0 200 OK
Server: Apache
Last-Modified: Sun, 22 Jul 2012 22:33:50 GMT
ETag: "841a4-48-b9250f80"
Content-Type: text/plain
Cache-Control: max-age=15
Expires: Sun, 22 Jul 2012 22:34:15 GMT
Date: Sun, 22 Jul 2012 22:34:00 GMT
Content-Length: 72
Connection: close
X-Akamai-Edgescape: country_code=US

{"ss":[["Sun","8:00","Pregame",,"ARI",,"NO",,,,"55760",,"PRE0","2012"]]}
disconnecting.

Now I just need to filter the header text, but that will be easy.

robertor · Aug 05, 2012, 04:57 am

Did you get to parse de json?

Greets,
Roberto

boelle · Nov 23, 2012, 11:12 am

would also like to know....

i can use this in a current project where a page returns raw text

Topic: Using Ethernet Shield to read text from a web page (Read 19585 times) previous topic - next topic

DPax

PaulS

DPax

robertor