[Solved] How to process multi-frame HTTP requests? (W5100 + 2560 + Ethernet lib)

Hi!

I am writing an HTTP server and a webpage and one of it’s features is to receive a PUT or POST request and write the body of the request to a file. Problem: It only writes the first Ethernet frame’s content to the file, then the EthernetClient library says there is no more data (client.available() is false).
The server can send multi-frame responses without any problems.
I also tried this with a pre-written HTTP server library (Webduino), it works the same.
So the questions are: What causes the problem? (My code?/The Ethernet library?/The Ethernet Shield?/Something entirely else?)
What can I do to make it work? Do I have to make a workaround? (I would like to avoid that)

Hardware: Arduino Mega 2560 + Ethernet Shield (with W5100)
Libraries: Ethernet, SdFat

Code of the server:

#include <Ethernet.h>
#include <SdFat.h>

const byte macAddress[] = { 0x02, 0x52, 0x3B, 0xE1, 0xCA, 0x19 };

const IPAddress IPAddressOfServer(192, 168, 1, 5);
const IPAddress networkMask(255, 255, 255, 255);
const IPAddress defaultGateway(192, 168, 1, 1);
const IPAddress DNSServer(8, 8, 8, 8);

EthernetServer server(80);
EthernetClient client;
SdFat SD;
File file; // this is OK

void ProcessRequest()
{
 // example:
 // GET /asd.txt HTTP/1.1

 // "GET"
 char method[5];
 uint16_t index = 0;
 char temp = client.read();
 while (index < 4 && temp != ' ') {
 method[index++] = temp;
 temp = client.read();
 }
 method[index] = '\0';
 
 // "/asd.txt"
 char url[32];
 index = 0;
 temp = client.read();
 while (index < 31 && temp != ' ') {
 url[index++] = temp;
 temp = client.read();
 }
 url[index] = '\0';

 // if the client wants to read a file
 if (!strcmp(method, "GET")) {

 Serial.println("GET!");

 if (SD.exists(url)) {
 // the file requested is present

 client.println("HTTP/1.1 200 OK");
 client.println("Content - Type: text/plain\n");

 // send the file to the client
 file = SD.open(url);
 while (file.available()) {
 client.write(file.read());
 }

 file.close();
 }
 else {
 // the file is not present
 // send 404 to client

 client.println("HTTP/1.1 404 Not Found");
 client.println("Content - Type: text/plain\n");
 client.print("<h1>404 :(</h1>");
 }
 }

 // if the client wants to put (rewrite/create) a file
 // at this point we also write most of the HTTP request to the file, but that's OK now
 if (!strcmp(method, "POST"))
 {
 Serial.println("POST!");

 // if the file exists, we open it
 if (SD.exists(url)) {

 file = SD.open(url, O_WRITE);

 // and overwrite it
 while (client.available()) {
 file.write(client.read());
 }

 // then truncate it
 file.truncate(file.position());
 // then close it
 file.close();

 // then send a success message
 client.println("HTTP/1.1 200 OK");
 client.println("Content - Type: text/plain\n");
 // and some content
 client.print("OK");

 }
 // otherwise we send a fail message
 else {

 client.println("HTTP/1.1 404 Not Found");
 client.println("Content - Type: text/plain\n");
 client.print("<h1>404 :(</h1>");
 }
 }
}

void setup() {

 Serial.begin(921600);

 Ethernet.begin(macAddress, IPAddressOfServer,
 DNSServer, defaultGateway, networkMask);

 SD.begin(4);

 server.begin();

 Serial.println("started!");

 delay(100); // Give all the stuff time to get ready
}

void loop() {

 client = server.available();

 if (client) {
 Serial.println("Incoming request!");

 ProcessRequest();
 client.stop();
 }

 delay(100);
}

Thank you for reading, any help is appreciated!

G.

My server example will upload almost any file requested. http://playground.arduino.cc/Code/WebServerST

SurferTim: My server example will upload almost any file requested. http://playground.arduino.cc/Code/WebServerST

I'm sorry, but I don't see in your code how you upload a file to your server, could you point me to the appropriate part of your code?

G.

czvendel:
Hi!

I am writing an HTTP server and a webpage and one of it’s features is to receive a PUT or POST request and write the body of the request to a file. Problem: It only writes the first Ethernet frame’s content to the file, then the EthernetClient library says there is no more data (client.available() is false).
The server can send multi-frame responses without any problems.
I also tried this with a pre-written HTTP server library (Webduino), it works the same.
So the questions are: What causes the problem? (My code?/The Ethernet library?/The Ethernet Shield?/Something entirely else?)
What can I do to make it work? Do I have to make a workaround? (I would like to avoid that)

Hardware: Arduino Mega 2560 + Ethernet Shield (with W5100)
Libraries: Ethernet, SdFat

Code of the server:

#include <Ethernet.h>

#include <SdFat.h>

const byte macAddress = { 0x02, 0x52, 0x3B, 0xE1, 0xCA, 0x19 };

const IPAddress IPAddressOfServer(192, 168, 1, 5);
const IPAddress networkMask(255, 255, 255, 255);
const IPAddress defaultGateway(192, 168, 1, 1);
const IPAddress DNSServer(8, 8, 8, 8);

EthernetServer server(80);
EthernetClient client;
SdFat SD;
File file; // this is OK

void ProcessRequest()
{
// example:
// GET /asd.txt HTTP/1.1

// “GET”
char method[5];
uint16_t index = 0;
char temp = client.read();
while (index < 4 && temp != ’ ') {
method[index++] = temp;
temp = client.read();
}
method[index] = ‘\0’;

// “/asd.txt”
char url[32];
index = 0;
temp = client.read();
while (index < 31 && temp != ’ ') {
url[index++] = temp;
temp = client.read();
}
url[index] = ‘\0’;

// if the client wants to read a file
if (!strcmp(method, “GET”)) {

Serial.println(“GET!”);

if (SD.exists(url)) {
// the file requested is present

client.println(“HTTP/1.1 200 OK”);
client.println(“Content - Type: text/plain\n”);

// send the file to the client
file = SD.open(url);
while (file.available()) {
client.write(file.read());
}

file.close();
}
else {
// the file is not present
// send 404 to client

client.println(“HTTP/1.1 404 Not Found”);
client.println(“Content - Type: text/plain\n”);
client.print(“

404 :frowning:

”);
}
}

// if the client wants to put (rewrite/create) a file
// at this point we also write most of the HTTP request to the file, but that’s OK now
if (!strcmp(method, “POST”))
{
Serial.println(“POST!”);

// if the file exists, we open it
if (SD.exists(url)) {

file = SD.open(url, O_WRITE);

// and overwrite it
while (client.available()) {
file.write(client.read());
}

// then truncate it
file.truncate(file.position());
// then close it
file.close();

// then send a success message
client.println(“HTTP/1.1 200 OK”);
client.println(“Content - Type: text/plain\n”);
// and some content
client.print(“OK”);

}
// otherwise we send a fail message
else {

client.println(“HTTP/1.1 404 Not Found”);
client.println(“Content - Type: text/plain\n”);
client.print(“

404 :frowning:

”);
}
}
}

void setup() {

Serial.begin(921600);

Ethernet.begin(macAddress, IPAddressOfServer,
DNSServer, defaultGateway, networkMask);

SD.begin(4);

server.begin();

Serial.println(“started!”);

delay(100); // Give all the stuff time to get ready
}

void loop() {

client = server.available();

if (client) {
Serial.println(“Incoming request!”);

ProcessRequest();
client.stop();
}

delay(100);
}




Thank you for reading, any help is appreciated!

G.

change this part of your code,

// and overwrite it
 while (client.available()) {
 file.write(client.read());
 }

to

 // and overwrite it
 unsigned long timeOut = millis(); // create a timeout 

 while(client.connected()&&(millis()-timeOut<10000)){
   if(client.available()){
     file.write(client.read());
     timeOut = millis(); // restart 10 second timeout
     }
   }

 if(client.connected()&&(millis()-timeOut>=10000)){ // connection stalled for at least 10 seconds
   Serial.print("Connection Stalled after ");
   Serial.print(file.position(),DEC);
   Serial.println(" bytes received.");
   Serial.println("upload Aborted!/ended?");
   }
 else if(!client.connected()){
   Serial.print("Connection lost after ");
   Serial.print(file.position(),DEC);
   Serial.println(" bytes received.");
   Serial.println("upload Aborted!");
   }

the way you are handling the POST command, you cannot identify when the POST upload has completed successfully. You need to review the HTTP POST protocol.

lookup Content-length

Chuck.

chucktodd:
change this part of your code,

// and overwrite it

while (client.available()) {
file.write(client.read());
}




to



// and overwrite it
unsigned long timeOut = millis(); // create a timeout

while(client.connected()&&(millis()-timeOut<10000)){
  if(client.available()){
    file.write(client.read());
    timeOut = millis(); // restart 10 second timeout
    }
  }

if(client.connected()&&(millis()-timeOut>=10000)){ // connection stalled for at least 10 seconds
  Serial.print(“Connection Stalled after “);
  Serial.print(file.position(),DEC);
  Serial.println(” bytes received.”);
  Serial.println(“upload Aborted!/ended?”);
  }
else if(!client.connected()){
  Serial.print(“Connection lost after “);
  Serial.print(file.position(),DEC);
  Serial.println(” bytes received.”);
  Serial.println(“upload Aborted!”);
  }




the way you are handling the POST command, you cannot identify when the POST upload has completed successfully. You need to review the HTTP POST protocol.

lookup Content-length

Chuck.

Thank you very much, this code just solved my problems in 5 seconds! Kinda funny, I’ve tried something similar, but without deep knowledge of the HTTP protocol, I was just shooting in the dark. I’ll make sure to check on the POST and GET methods!
Also, a separate thank you for the personalized code, I just had to paste it, and it worked! You are awesome!

G.

Hello guys,

I'm new to arduino and don't know much about HTTP requests, seems you figured out your problem with the requests, can any of you guys help me or give me some advices, I'm trying to make a very simple POST request using Arduino YUN without success, see below;

https://forum.arduino.cc/index.php?topic=491828.0