File download with Arduino web server?

Just a thought. If I have Arduino server loaded with web pages, why not having a few links on the web pages that point to several data log files for download?

My question is, how to? I have some rough ideas so I appreciate it if experts can help me polish the ideas before I go ahead and code it (into non-working result of course).

Arduino web server has a web page and a link on the page say:

Download data log.

The upon getting a GET method like the following, Arduino will confirm the file name exists on the SD card:

GET /datalog.txt HTTP/1.1

Arduino server then writes a response like:

something something
Content-Type: text/csv
Connection: close

1234, 567, 890 (log file content)
2345, 678, 901

(end of transmission)

Will this be sufficient? Thank you!

What code base do you use for your web server? Most web server code serving data from the SD card already includes your needed features with the exception that they don't select the text/csv MIME type if a *.txt file is chosen.

pylon:
What code base do you use for your web server? Most web server code serving data from the SD card already includes your needed features with the exception that they don't select the text/csv MIME type if a *.txt file is chosen.

pylon,

I am writing code to make my own web server on arduino MEGA so I don't know what code base. Do you mean some sort of encoding binary into alphanumerical code? This needs to be integrated into an existing project so I prefer writing my own code and make it only as powerful as needed not to be capable of processing other things that I don't need. My goal is to put a link on my arduino-generated web page. If it gets clicked, arduino will reply with the file, like if you click on a .csv file link on this webpage (look below) and your browser asks you where to store that file or maybe just displays it on the browser screen. I wanted to know whether my naive thinking is correct.

fcm-eddm-retail.csv (1019 Bytes)

My initial tests are positive. I can make Arduino serve files to a web client (firefox browser). But the speed is very slow around 70-80 bytes per second. I was reading file from SD card and sending via wifi shield. I'll have to find the reason of this slow speed or it's not practical.

Code that increases the sent packet size from an SD card.

//zoomkat 12/26/12
//SD server test code
//open serial monitor to see what the arduino receives
//address will look like http://192.168.1.102:84 when submited
//for use with W5100 based ethernet shields

#include <SD.h>
#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 
  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; //physical mac address
byte ip[] = { 
  192, 168, 1, 102 }; // ip in lan
byte gateway[] = { 
  192, 168, 1, 1 }; // internet access via router
byte subnet[] = { 
  255, 255, 255, 0 }; //subnet mask
EthernetServer server(84); //server port
String readString; 

//////////////////////

void setup(){

  Serial.begin(9600);

  // disable w5100 while setting up SD
  pinMode(10,OUTPUT);
  digitalWrite(10,HIGH);
  Serial.print("Starting SD..");
  if(!SD.begin(4)) Serial.println("failed");
  else Serial.println("ok");

  Ethernet.begin(mac, ip, gateway, gateway, subnet);
  //delay(2000);
  server.begin();
  Serial.println("Ready");
}

void loop(){
  // Create a client connection
  EthernetClient client = server.available();
  if (client) {
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();

        //read char by char HTTP request
        if (readString.length() < 100) {
          //store characters to string 
          readString += c; 
          //Serial.print(c);
        } 
        //if HTTP request has ended
        if (c == '\n') {

          ///////////////
          Serial.println(readString); //print to serial monitor for debuging 

            client.println("HTTP/1.1 200 OK"); //send new page
          client.println("Content-Type: text/html");
          //client.println("Content-Type: image/jpeg");
          //client.println("Content-Type: image/gif");
          //client.println("Content-Type: application/x-javascript");
          //client.println("Content-Type: text");

          client.println();

          //File myFile = SD.open("boom.htm");
          File myFile = SD.open("HYPNO.JPG");
          //File myFile = SD.open("BLUEH_SL.GIF");
          //File myFile = SD.open("SERVOSLD.HTM");

          if (myFile) {

            byte clientBuf[64];
            int clientCount = 0;

            while(myFile.available())
            {
              clientBuf[clientCount] = myFile.read();
              clientCount++;

              if(clientCount > 63)
              {
                // Serial.println("Packet");
                client.write(clientBuf,64);
                clientCount = 0;
              }
            }
            //final <64 byte cleanup packet
            if(clientCount > 0) client.write(clientBuf,clientCount);            
            // close the file:
            myFile.close();
          }
          delay(1);
          //stopping client
          client.stop();
          readString="";
        }
      }
    }
  } 
}

But the speed is very slow around 70-80 bytes per second. I was reading file from SD card and sending via wifi shield. I'll have to find the reason of this slow speed or it's not practical.

My guess is (as was the guess of zoomkat) that you're sending one complete packet for every character you want to transmit. You get a much higher throughput if you take care (by not using the print/println methods but the write(ptr, size) method) that packets contain more than one byte of data.

If you show us your code we may be able to help you.

pylon:

But the speed is very slow around 70-80 bytes per second. I was reading file from SD card and sending via wifi shield. I'll have to find the reason of this slow speed or it's not practical.

My guess is (as was the guess of zoomkat) that you're sending one complete packet for every character you want to transmit. You get a much higher throughput if you take care (by not using the print/println methods but the write(ptr, size) method) that packets contain more than one byte of data.

If you show us your code we may be able to help you.

I bet I was!

My essential code:

while(myFile.available()
{
  client.write(myFile.read());
}

After doing something similar to zoomkat's code, I was able to get about 20KB/s (hard to measure unless I send a much larger file)

I noticed the client.print in wifi class has a size limit of around 90 bytes. Beyond the limit if you print a string of say 100 bytes, nothing is received.

Zoomkat,

Is the 64 just the buffer size (small to save SRAM) or is it tied to limitation of Ethernet hardware? Thanks.

Is the 64 just the buffer size (small to save SRAM) or is it tied to limitation of Ethernet hardware? Thanks.

I didn't write it, but I think the size was chosen to not challenge the SRAM limitations.

Hello

I had a project of creating an Arduino weather station. With a litle of research i found the solution that works great:

First, you have to standardly check, if client is conected and if there is a request (standard webserver code)

On your web page, you have a link to LOG.txt file, which is saved on SD card. link to file is standard link

After the click, server recieves "GET/ LOG.txt"

Server response is the following

client.println("HTTP/1.1 200 OK");
client.println("Content-Disposition: attachment; filename=LOG.txt");
client.println("Connection: close");
client.println();

webFile = SD.open("LOG.txt"); // open file

if (webFile)
{
while(webFile.available())
{
client.write(webFile.read()); // send file to client
}
webFile.close();
}

delay(1); // give the web browser time to receive the data
client.stop(); // close the connection

Hi,

I know it's old topic but exactly what i'm looking for
I'm new on Arduino.
First, sorry for my english.
I would like to dowload a .txt file stored in the SD Card by a local network.

I use Arduino UNO, Ethernet Shield with Wiznet W5500.
my .txt file called TEST.txt

With your good informations, i did the following code,
but when i put the IP address in my web browser, I have an automatic dowload of a file called "TEST.txt" but it is totally empty!
I don't understand why I haven't my data in my file.

Thank you for your help

#include <SD.h>
#include <SPI.h>
#include <Ethernet2.h>

#define REQ_BUF_SZ 20
byte mac[] = {0x90, 0xA2, 0xDA, 0x10, 0xA5, 0xC0};
IPAddress ip(169, 254, 155, 166);
byte gateway []={169, 254, 155, 1};
byte subnet []={255, 255, 0, 0};
EthernetServer server(80);
File myFile;
char HTTP_req[REQ_BUF_SZ] = {0}; // buffered HTTP request stored as null terminated string
char req_index = 0;
String readString;

//////////////////////

void setup()
{
pinMode(10, OUTPUT);
digitalWrite(10, HIGH);
Serial.begin(9600);
// initialize SD card
Serial.println("Initializing SD card...");
if (!SD.begin(4))
{
Serial.println("ERROR - SD card initialization failed!");
return;
}
Serial.println("SUCCESS - SD card initialized.");
if (!SD.exists("TEST.txt"))
{
Serial.println("ERROR - Can't find index.htm file!");
return; // can't find index file
}
Serial.println("SUCCESS - Found TEST.txt file.");
Ethernet.begin(mac, ip, gateway, subnet); // initialize Ethernet device
server.begin(); // start to listen for clients
Serial.println("Ready");
}

void loop()
{
// Create a client connection
EthernetClient client = server.available();
if (client)
{
boolean currentLineIsBlank = true;
while (client.connected())
{
if (client.available())
{
char c = client.read();
//read char by char HTTP request
if (readString.length() < 100)
{
//store characters to string
readString += c;
Serial.print(c);
}
//if HTTP request has ended
if (c == '\n' && currentLineIsBlank)
{
// open requested web page file
client.println();
if (HTTP_req, "GET /TEST.txt")
{
client.println("HTTP/1.1 200 OK"); //send new page
client.println("Content-Disposition: attachment; filename=TEST.txt");
client.println("Connection: close");
client.println();
Serial.println(readString); //print to serial monitor for debuging
if (myFile)
{
byte clientBuf[64];
int clientCount = 0;
while(myFile.available())
{
clientBuf[clientCount] = myFile.read();
clientCount++;
if(clientCount > 63)
{
client.write(clientBuf,64);
clientCount = 0;
}
}
//final <64 byte cleanup packet
if(clientCount > 0) client.write(clientBuf,clientCount);
// close the file:
myFile.close();
}
delay(1);
//stopping client
client.stop();
readString="";
}
}
}
}
}
}