Arduino Wifi Shield download file GET

Hello!

I was looking around all day.
I've been trying to send a file through http get, using the wifi shield.
I started of by using the code from harizanov.com:

/*
 
 Modified by harizanov.com to fetch a .bin file from Internet and save to SD card
 
 DNS and DHCP-based Web client
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13
 
 created 18 Dec 2009
 by David A. Mellis
 modified 12 April 2011
 by Tom Igoe, based on work by Adrian McEwen

 
 
 */

#include <SPI.h>
#include <HttpClient.h>
#include <SD.h>
File myFile;
#include <WiFi.h>
// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0xaa, 0xaa, 0xaa, 0xaa, 0x07, 0x23 };
char serverName[] = "llalaa.com";


char ssid[] = "XXX"; //  your network SSID (name) 
char pass[] = "XXXX";    // your network password (use for WPA, or use as key for WEP)
int keyIndex = 0;            // your network key Index number (needed only for WEP)
int status = WL_IDLE_STATUS;
WiFiClient client;
void printWifiStatus() {
  // print the SSID of the network you're attached to:
  Serial.print("- SSID: ");
  Serial.println(WiFi.SSID());

  // print your WiFi shield's IP address:
  IPAddress ip = WiFi.localIP();
  Serial.print("-IP Address: ");
  Serial.print(ip);
  Serial.println("-");

  // print the received signal strength:
  long rssi = WiFi.RSSI();
  Serial.print("-Signal Str (RSSI): ");
  Serial.print(rssi);
  Serial.println(" dBm-");
}
void setup() {
  // start the serial library:
  Serial.begin(9600);
  
    Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin 
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output 
  // or the SD library functions will not work. 
   pinMode(53, OUTPUT);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
  
  
  
  // start the Ethernet connection:
 while ( status != WL_CONNECTED) { 
	Serial.println("- Setting WiFi Connection  -");
        			
    status = WiFi.begin(ssid, pass);
    // wait 10 seconds for connection:
    delay(2500);
  } 
	Serial.println("-     Connected to WiFi    -");
	printWifiStatus();
  
  

  // if you get a connection, report back via serial:
  
  if (client.connect(serverName, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /testing.txt HTTP/1.1");
    client.println();
    readPastHeader(&client);
  } 
  else {
    // you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}
void loop()
{
  
   delay(1999);
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {

  // Check to see if the file exists: 
  if (SD.exists("music.txt")) {
    Serial.println("file exists, deleting..");
    SD.remove("music.txt");
  }
  else {
    Serial.println("file doesn't exist.");  
  }
 
    myFile = SD.open("music.txt", FILE_WRITE);
      
    if (myFile) {
    Serial.print("Writing to file...");
    
      while (client.connected()) {
        byte c = client.read();
        Serial.write('.');
        myFile.write(c);
      }
    
      // close the file:
      myFile.close();
      Serial.println("");
      Serial.println("done.");

	  /*
	  cli();
	  asm volatile("jmp 0x7000");
	  */
      
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening file");
  }  
      

  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}


bool readPastHeader(Client *pClient)
{
  bool bIsBlank = true;
  while(true)
  {
    if (pClient->available()) 
    {
      char c = pClient->read();
      if(c=='\r' && bIsBlank)
      {
        // throw away the /n
        c = pClient->read();
        return true;
      }
      if(c=='\n')
        bIsBlank = true;
      else if(c!='\r')
        bIsBlank = false;
      }
   }
}

the name of my server happens to be my question. What should it be? I tried using my own website at webs.com, but it doesnt work, it starts a client, but creates an empty file in SD card.

the file is acessible at lisandrolopes.webs.com/datalog.txt .Thats the right way that i have to refer? to put in the servername the "lisandrolopes.webs.com" and /datalog in the GET string?.
Is it possible to download it and saving it to SD card? my goal is to then upload an hex file from SD and run new sketch indicated there. How can i get info about what type of HTTP a website uses? if i have to use 1.0, 1.1?

Any guidance or explanation highly appreciated, thanks!

the name of my server happens to be my question. What should it be?

Don't you know the name of your server? If not, how are we to be expected to know it?

I tried using my own website at webs.com, but it doesnt work, it starts a client, but creates an empty file in SD card.

If that is a multi-domain hosting company, you need a Host: statement after the GET statement, to define the specific domain.

Thanks for answering Paul. I didnt explained myself correctly. I meant the array of chars named serverName. If my file is located at lisandrolopes.webs.com/datalog.txt, how should i separate it through the serverName and the statement that goes after the GET command?

Lis:
Thanks for answering Paul. I didnt explained myself correctly. I meant the array of chars named serverName. If my file is located at lisandrolopes.webs.com/datalog.txt, how should i separate it through the serverName and the statement that goes after the GET command?

If you're asking what needs to be in the HTTP GET request, it would need to include something like:

GET /datalog.txt HTTP/1.1
Host: lisandrolopes.webs.com

That would correspond to the URL:
http://lisandrolopes.webs.com/datalog.txt

Well, thats how i was doing it, and it wouldnt write anything to file. I changed to another webserver, hosted at 000webhost.com. I pinged my website lislog.webatu.com, got the IP adress and ran the following:

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
File myFile;

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0x90, 0xA2, 0xDA, 0x0D, 0x3E, 0xF8 };
IPAddress server(31,170,161,76); 

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  
    Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin 
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output 
  // or the SD library functions will not work. 
   pinMode(10, OUTPUT);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
  
  
  
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);  
  
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET public_html/music.txt HTTP/1.1");
    client.println();
    readPastHeader(&client);
  } 
  else {
    // you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
   delay(2000);
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {

  // Check to see if the file exists: 
  if (SD.exists("datalog.txt")) {
    Serial.println("file exists, deleting..");
    SD.remove("datalog.txt");
  }
  else {
    Serial.println("file doesn't exist.");  
  }
 
    myFile = SD.open("datalog.txt", FILE_WRITE);
      
    if (myFile) {
    Serial.print("Writing to file...");
    
      while (client.connected()) {
        byte c = client.read();
        Serial.write('.');
        myFile.write(c);
      }
    
      // close the file:
      myFile.close();
      Serial.println("");
      Serial.println("done.");

	  /*
	  cli();
	  asm volatile("jmp 0x7000");
	  */
      
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening file");
  }  
      

  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}


bool readPastHeader(Client *pClient)
{
  bool bIsBlank = true;
  while(true)
  {
    if (pClient->available()) 
    {
      char c = pClient->read();
      if(c=='\r' && bIsBlank)
      {
        // throw away the /n
        c = pClient->read();
        return true;
      }
      if(c=='\n')
        bIsBlank = true;
      else if(c!='\r')
        bIsBlank = false;
      }
   }
}

It did print out something in the file output. But I'm guessing the problem is from the website, since i can't access the file through it.
Any ideas about places to host a file and that can be used in this setup?

It did print out something in the file output. But I'm guessing the problem is from the website, since i can't access the file through it.

But, you're not going to tell us what that ways, I guess. So, all we can do is guess along with you. I'd guess that there is nothing wrong with any web site anywhere, since no web sites are involved. Scripts on web servers that might be accessed by a web browser, yes.

This is what I'm trying to do. Trying to put new file in SD card.
File is here: http://lislog.webatu.com/download/testing.pdf -its an example of a file

so I did this:

/*
 
 Modified by harizanov.com to fetch a .bin file from Internet and save to SD card
 
 DNS and DHCP-based Web client
 Circuit:
 * Ethernet shield attached to pins 10, 11, 12, 13
 
 created 18 Dec 2009
 by David A. Mellis
 modified 12 April 2011
 by Tom Igoe, based on work by Adrian McEwen

 
 
 */

#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
File myFile;

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = {  0x90, 0xA2, 0xDA, 0x0D, 0x3E, 0xF8 };
IPAddress server(31,170,161,76); 

// Initialize the Ethernet client library
// with the IP address and port of the server 
// that you want to connect to (port 80 is default for HTTP):
EthernetClient client;

void setup() {
  // start the serial library:
  Serial.begin(9600);
  
    Serial.print("Initializing SD card...");
  // On the Ethernet Shield, CS is pin 4. It's set as an output by default.
  // Note that even if it's not used as the CS pin, the hardware SS pin 
  // (10 on most Arduino boards, 53 on the Mega) must be left as an output 
  // or the SD library functions will not work. 
   pinMode(10, OUTPUT);
   
  if (!SD.begin(4)) {
    Serial.println("SD initialization failed!");
    return;
  }
  Serial.println("SD initialization done.");
  
  
  
  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    while(true);
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);  
  
  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  
  if (client.connect(server, 80)) {
    Serial.println("connected");
    // Make a HTTP request:
    client.println("GET /download/testing.pdf HTTP/1.1");
    client.println();
    readPastHeader(&client);
  } 
  else {
    // you didn't get a connection to the server:
    Serial.println("connection failed");
  }
}

void loop()
{
   delay(2000);
  // if there are incoming bytes available 
  // from the server, read them and print them:
  if (client.available()) {

  // Check to see if the file exists: 
  if (SD.exists("datalog.txt")) {
    Serial.println("file exists, deleting..");
    SD.remove("datalog.txt");
  }
  else {
    Serial.println("file doesn't exist.");  
  }
 
    myFile = SD.open("datalog.txt", FILE_WRITE);
      
    if (myFile) {
    Serial.print("Writing to file...");
    
      while (client.connected()) {
        byte c = client.read();
        Serial.write('.');
        myFile.write(c);
      }
    
      // close the file:
      myFile.close();
      Serial.println("");
      Serial.println("done.");

	  /*
	  cli();
	  asm volatile("jmp 0x7000");
	  */
      
  } else {
    // if the file didn't open, print an error:
    Serial.println("error opening file");
  }  
      

  }

  // if the server's disconnected, stop the client:
  if (!client.connected()) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();

    // do nothing forevermore:
    while(true);
  }
}


bool readPastHeader(Client *pClient)
{
  bool bIsBlank = true;
  while(true)
  {
    if (pClient->available()) 
    {
      char c = pClient->read();
      if(c=='\r' && bIsBlank)
      {
        // throw away the /n
        c = pClient->read();
        return true;
      }
      if(c=='\n')
        bIsBlank = true;
      else if(c!='\r')
        bIsBlank = false;
      }
   }
}

and it wrote to file txt this. I tried with other files extensions but nothing.
I'm not expecting to have a proper format output in the sd card, since im saving it into a txt file, but i want to receive something...

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.

</p>
</body></html>

Any ideas on what I am doing wrong?
Thanks

Any ideas on what I am doing wrong?

The server is telling you exactly what the problem is.

lislog.webatu.com is a multi-domain hosting site.
You need to include a specific Host: statement, using client.println(), after the GET request, before the blank line.

I don't understand what you mean by statement. This is the info that I have on the website:

Domain lislog.webatu.com
Username a7112518
Password ******
Disk Usage 0.02 / 1500.0 MB
Bandwidth 100000 MB (100GB)
Home Root /home/a7112518
Server Name server37.000webhost.com

And

Nameserver details (only if you host your own domain)

ns01.000webhost.com 31.170.167.6
ns02.000webhost.com 31.170.164.248

You mean any of this information?

Now, It actually communicated with the server. With the same file of the example, it does exchange data to SD card, though the file is smaller than the original one. From 1260 bytes, I have 2XX'ish in the SD card.

Why do so many older examples have blank #include files? That's really frustrating.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.