Hello everybody.
I'm working on a simple webserver using Arduino MKR1010 WIFI.
My sketch is based on the "WifiWebServerSketch" that you can find in the examples of WiFiNINA library.
I analize the GET request received from the client and I sent him the response.
If the client requests a file present in the SD card, then I use a function (written by myself) that uses a buffer to read the file from the SD card and send it to the client using the client.write(buf, len) function, in order to speed-up the transmission (as known, reading and sending 1 single byte at a time is a waste of time).
I use the number of bytes written from the instruction client.write to check if the transmission has been succesfully completed.
My problem is this: sometimes the client can receive the complete file and sometimes the file transmission stops randomly at some point (in this case I can even try to send again the content of my buffer but it doesn't help and the instruction client.write keeps returning zero bytes transmitted, so I abort the transmission).
I tryed this for small (some KBytes) or "big" (some MBytes) files but I get the same problem.
Am I forgetting something? Is there somebody that got the same problem in similar application?
do you set content-length header?
esp8266 has client.write(file). ESP32 doesn't have it?
Juraj:
do you set content-length header?
esp8266 has client.write(file). ESP32 doesn't have it?
No, up to now I send always the same header without sending the content-length field. I will try to send the correct header and tell if it works.
dario_show:
No, up to now I send always the same header without sending the content-length field. I will try to send the correct header and tell if it works.
Adding the content-length header (like in your code) didn't solved completely the problem, even if now seems better.
I still got the transmission stopped randomly in some cases during these last days.
Using Chrome tools I can see that when it stops, I get the error "ERR_CONTENT_LENGTH_MISMATCH 200 (OK)" for my GET request and the write instruction returns less byte than expected.
Anyway I'm still learning how HTTP transmission works and I need to find some good guide for how to create a complete and correct header from arduino.
But it sounds strange to me that most of the times the transfer completes without errors or problems and sometimes stops
Can you suggest me an example sketch using data transfer with buffer?
show your sketch
Ok, I will need some time to cut a lot of things and make the sketch minimal so it will be easier to understand and debug.
But what happen and what should I do when the "client.write(buf, len)" returns less bytes than "len"?
dario_show:
Ok, I will need some time to cut a lot of things and make the sketch minimal so it will be easier to understand and debug.
But what happen and what should I do when the "client.write(buf, len)" returns less bytes than "len"?
process len bytes and then read again. or use readBytes. it will wait for bytes
I try to send only the part of code where I get the problem, I think the complete code is too long and not very neat for now. This function takes as argument a file path and send it to the client. It uses an array as a buffer.
For example if SD_BUFFER_SIZE = 10 and client.write sends only 7 bytes, then I wait for 5ms and then try again to transmit the last 3 bytes from the buffer.
Changing SD_BUFFER_SIZE or the delay time doesn't help, 95% of the times the file transfer completes
without problems, the other times it stops in a random point. I would like to achieve a more reliable comunication.
P.S. up to now I can download files from Arduino at about 100 kbyte/s
#define SD_BUFFER_SIZE 1460
char fileBuffer[SD_BUFFER_SIZE];
void sendFileFromSD(String filePath) {
File fileToSend = SD.open(filePath, FILE_READ);
unsigned long FileSize, fileSendStartTime, totalByteSent;
int totalRetryCount = 0;
if (fileToSend) {
FileSize = fileToSend.size();
fileSendStartTime = millis();
// BEGIN OF HEADER
client.println(F("HTTP/1.1 200 OK"));
client.print(F("Content-Length: "));
client.println(FileSize);
client.println(F("Connection: close"));
client.print(F("Content-Type: "));
switch (clientRequestType) {
case FILE_HTM:
client.println(F("text/html"));
break;
case FILE_TXT:
client.println(F("text/plain"));
break;
case FILE_JPG:
client.println(F("image/jpeg"));
break;
case FILE_PNG:
client.println(F("image/png"));
break;
case FILE_JAVASCRIPT:
client.println(F("text/javascript"));
break;
default:
client.println(F("text/plain"));
}
client.println(); // empty line to close header
// END OF HEADER
totalByteSent = 0;
boolean readError = false, writeError = false;
while ((totalByteSent < FileSize) && (!readError) && (!writeError)) {
int bufferLength = SD_BUFFER_SIZE; // init with buffer max size
if (bufferLength > FileSize - totalByteSent)
bufferLength = FileSize - totalByteSent;
int partialByteRead = fileToSend.read(fileBuffer, bufferLength);
if (partialByteRead < bufferLength) {
readError = true;
}
else {
int SendCount = 0; // first send attempt
int bufferByteSent = 0;
while ((bufferByteSent < bufferLength) && (!writeError)) { // until all reads byte are not sent ...
int remainingByteToSend = bufferLength - bufferByteSent;
int partialByteSent = client.write(&fileBuffer[bufferByteSent], remainingByteToSend);
bufferByteSent += partialByteSent;
totalByteSent += partialByteSent;
if (partialByteSent < remainingByteToSend)
delay(5); // wait some time if transfer has not been completed
SendCount++;
if (SendCount > 20) // maximum 20 times then abort
writeError = true;
if (SendCount > 1)
totalRetryCount++;
}
}
}
fileToSend.close();
}
}
I begin thinking nobody is using Arduino for file transfer.
I wish Arduino documentation could be better and could explain what to do in case the transfer stops. Maybe is only a bug?
Even the function client.flush() is not clear to me: when should it be called?
dario_show:
I begin thinking nobody is using Arduino for file transfer.
I wish Arduino documentation could be better and could explain what to do in case the transfer stops. Maybe is only a bug?
Even the function client.flush() is not clear to me: when should it be called?
and see how simple is the file serving in my WebServer with BufferedPrint
Thank-you for your help, Juraj.
Sure, I'm not expert but I'm trying to write my code only using the WiFiNINA library, hoping that all its functions are well tested.
Anyway, I opened the source code of the library and found an interesting thing: the code of the flush() function is this:
void WiFiClient::flush() {
- // TODO: a real check to ensure transmission has been completed*
}
So it seems this function is doing nothing, not as documentation says.
This makes me think the authors didn't complete this part (and maybe some others??) and I can't rely on this library
dario_show:
Thank-you for your help, Juraj.
Sure, I'm not expert but I'm trying to write my code only using the WiFiNINA library, hoping that all its functions are well tested.
Anyway, I opened the source code of the library and found an interesting thing: the code of the flush() function is this:void WiFiClient::flush() {
- // TODO: a real check to ensure transmission has been completed*
}So it seems this function is doing nothing, not as documentation says.
This makes me think the authors didn't complete this part (and maybe some others??) and I can't rely on this library
it is ok. they don't do buffering so there is nothing to flush.
use BufferedPrint for buffering