I have a ESP sensor which does log all system events to spiffs file (event log). I want to be able to open this file with a browser (to check for eventual system errors).
I have troubles sending this file to browser. It sends about 4-5 parts (each 1000bytes), and then it hangs. I guess there are too many parts sent in one session.
How would I send a long txt file to html page?
Thank you for your help!
Code is below:
void handleView_log()
{
int siz;
String oneline;
wdt_reset();
Serial.println (F("handleView_log"));
server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
server.sendHeader("Pragma", "no-cache");
server.sendHeader("Expires", "-1");
server.setContentLength(CONTENT_LENGTH_UNKNOWN); // *** BEGIN ***
server.send(200, "text/html", "");
s="";
s=s+F("<!doctype html><html><head><title></title></head><body><h1>Event log</h1><form target=\"_blank\">");
s=s+F("<table border=\"0\" cellpadding=\"1\" cellspacing=\"1\" style=\"width: 1200px;\"><tbody><p>");
server.sendContent(s);
log_file.close();
log_file = SPIFFS.open("/log.txt", "r");
if (!log_file)
{
s=s + F("<h1>Log file not found.</h1>"); // log.txt does not exist
}
else
{
if ( !log_file.size() )
{
s=s + F("<h1>Log file empty.</h1>"); // log.txt is empty
}
else
{
Serial.print F("Dumping log file size: "); // log.txt is valid for sending to browser
siz = log_file.size();
Serial.println (siz);
while (siz>0) // siz will count unread/unsent bytes
{
s = "";
while ( (s.length()<1000) & (siz>0) ) // read parts up to 1000bytes each
{
oneline = log_file.readStringUntil('\n');
s = s + "<p>" + oneline + "</p>"; // add html handles for each line
siz = siz - oneline.length();
}
server.sendContent (s); // send part
}
s = F("</p><p></p><p></p>END OF LOG");
}
}
s = s + F("</tbody></table>");
server.sendContent(s);
//Serial.println(s);
server.sendContent(""); // *** END 1/2 ***
server.client().stop(); // *** END 2/2 ***
return;
}
You have posted a code snippet. Code snippets can be a problem because they usually omit important information.
I would guess that the variable s is declared as String. String is problematic and to be avoided on small memory devices such as most Arduinos. String can lead to fragmented memory and hangs. C strings (text terminated by a null character) works well and avoids most hangs.
I am surprised that this even compiles:
Serial.print F("Dumping log file size: "); // log.txt is valid for sending to browser
I am away from the Arduino IDE and cannot try this on my mobile device.
I modified the code in a way it just reads chunks of file and prints millis and bytes remained to read:
void handleView_log()
{
int siz;
String oneline;
wdt_reset();
Serial.println (F("handleView_log"));
log_file.close();
log_file = SPIFFS.open("/log.txt", "r");
if (!log_file)
{
s=s + F("<h2>Log file not found.</h2>"); // log.txt does not exist
}
else
{
if ( !log_file.size() )
{
s=s + F("<h2>Log file empty.</h2>"); // log.txt is empty
}
else
{
Serial.print F("Dumping log file size: "); // log.txt is valid for sending to browser
siz = log_file.size();
Serial.println (siz);
while (siz>0) // siz will count unread/unsent bytes
{
s = "";
while ( (s.length()<1000) & (siz>0) ) // read parts up to 2000bytes each
{
oneline = log_file.readStringUntil('\n');
s = s + "<p>" + oneline + "</p>"; // add html handles for each line
siz = siz - oneline.length();
}
Serial.print (millis() );
Serial.print (" bytes left: ");
Serial.println (siz);
}
s = F("</p><p></p><p></p><h3>END OF LOG</h3>");
}
}
Serial.println(s);
return;
}
It is clearly that I have a problem with spiffs read. It works ok for few ten times (takes some 100ms per chunk), than is slows down and takes app. 15 seconds per chunk.
Below is the output, first number is millis(), second number is bytes remained:
Stupid. It makes NO sense to take a string literal that is stored in flash memory and copy it into a String instance in SRAM.
Quit pissing away memory on Strings. The read() method is overloaded to read n characters into a char array. The array can then be NULL-terminated and used wherever the data is (somewhere outside of your useless snippet) actually sent to the client.
I am using string 's' just for some kind of buffer, to collect info and then send in a packet, not sending each few bytes separate.
Also I when I am sending large HTML file, I use this string as a buffer, it consists of HTML code and variables, measuring results etc. I willingly sacrificed 2500 bytes of SRAM to have this buffer. I know it is not such a good idea, but still maybe is better this way, than using hundreds of calls to "server.sendContents (xxx).
I think I found the reason for above slow reading: I have to upgrade spiffs library to newer. I upgraded spiffs to 0.3.7.
After some code tweaking here is code that works:
void handleView_log()
{
int siz;
String oneline;
char kar;
wdt_reset();
Serial.println (F("handleView_log"));
server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
server.sendHeader("Pragma", "no-cache");
server.sendHeader("Expires", "-1");
server.setContentLength(CONTENT_LENGTH_UNKNOWN); // *** BEGIN ***
server.send(200, "text/html", "");
s = F("<!doctype html><html><h1>Event log</h1>");
server.sendContent(s);
log_file.close();
delay(100);
log_file = SPIFFS.open("/log.txt", "r");
if (!log_file)
{
server.sendContent F("<h2>Log file not found.</h2>"); // log.txt does not exist
}
else
{
if ( !log_file.size() )
{
server.sendContent F("<h2>Log file empty.</h2>"); // log.txt is empty
}
else
{
Serial.print F("Dumping log file size: "); // log.txt is valid for sending to browser
siz = log_file.size();
Serial.println (siz);
while (siz>0) // siz will count unread/unsent bytes
{
wdt_reset();
s = "";
while ( s.length()<2000 ) // read parts up to 2000bytes each
{
if (!log_file.available())
{
siz=0;
}
else
{
kar = log_file.read();
switch (kar)
{
case (10): s = s + "</p>";
break;
case (13): s = s + "<p>";
break;
default: s = s + kar;
}
siz--;
}
if (siz<=0) break;
}
Serial.print (millis() );
Serial.print (" bytes left: ");
Serial.println (siz);
server.sendContent (s); // send part
}
}
}
server.sendContent F("</p><p></p><p></p><h3>END OF LOG</h3>");
wdt_reset();
server.sendContent (""); // *** END 1/2 ***
server.client().stop(); // *** END 2/2 ***
return;
}
I am using string 's' just for some kind of buffer,
No. You are pissing away resources on Strings. They are NOT the same as strings, in any way, shape, or form.
to collect info and then send in a packet, not sending each few bytes separate.
You don't think the client is smart enough to do that?
I know it is not such a good idea, but still maybe is better this way, than using hundreds of calls to "server.sendContents (xxx).
Why would you need hundreds of them? One, called in a loop, sending the contents of the string (NOT String) of data read from the file is all that is needed.
I would gladly leave Strings, but have no idea how.
Here is a part of my code that serves html page. I am combining fixed text (HTML code) and measured values ( d[ x ].value ) and variables ( d[ x ].tag etc.). I don't know how else to do that, that's why I using String.
I use String s as a global, reserving a space of 2500. I assume by that, I sacrifized 2500 bytes of RAM for that kind of jobs in my project. Am I wrong?
Is there a way to see, what is "eating" my RAM, like some kind of RAM table?
I would appreciate if you can point me how to achieve above task without using Strings.
Here is the part of code (I did not attached whole HTML since it is very long:
server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
server.sendHeader("Pragma", "no-cache");
server.sendHeader("Expires", "-1");
server.setContentLength(CONTENT_LENGTH_UNKNOWN); // *** BEGIN ***
server.send(200, "text/html", "");
s="";
s=s+ F("<h4>Current measured values:</h4>");
s=s+F("<table><tbody>");
s=s+F("<tr><td>") + String(d[0].tag) + F("</td><td> </td><td>") + String(d[0].value) + ("</td><td> </td>");
s=s+F("<td>") + String(d[4].tag) + F("</td><td> </td><td>") + String(d[4].value)+F("</td></tr>");
s=s+F("<tr><td>") + String(d[1].tag) + F("</td><td> </td><td>") + String(d[1].value) + ("</td><td> </td>");
s=s+F("<td>") + String(d[5].tag) + F("</td><td> </td><td>") + String(d[5].value)+F("</td></tr>");
s=s+F("<tr><td>") + String(d[2].tag) + F("</td><td> </td><td>") + String(d[2].value) + ("</td><td> </td>");
s=s+F("<td>") + String(d[6].tag) + F("</td><td> </td><td>") + String(d[6].value)+F("</td></tr>");
s=s+F("<tr><td>") + String(d[3].tag) + F("</td><td> </td><td>") + String(d[3].value) + ("</td><td> </td>");
s=s+F("<td>") + String(d[7].tag) + F("</td><td> </td><td>") + String(d[7].value)+F("</td></tr>");
s=s+F("</table></tbody><p> </p>");
s=s+ F("<h4>Enter setup page:</h4>");
s=s+F("<table><tbody><tr><td>User</td> <td><input type=\"text\" name=\"username\"></td></tr>");
s=s+F("<tr><td>Password:</td> <td><input type=\"password\" name=\"password\"></td></tr></table>");
s=s+F("<p> </p><input type=\"submit\" value=\"Login\"></form> ");
server.sendContent(s);
..... some more HTML in String and sending with server.sendContent(s);
..... some more HTML in String and sending with server.sendContent(s);
....
server.sendContent(""); // *** END 1/2 ***
server.client().stop(); // *** END 2/2 ***
I suggest that you go to www.cplusplus.com and use the search box to look for the strcat(...) function. This function and others like it are available on the Arduino and make it easy to use c strings instead of String.
It would be easy for me to convert your code to use strcat(...) but I am hoping that you are willing to do some digging and get educated. There may be a little pain at first but in the long run it will be worthwhile.
At the start, the variable s is not a c string. The example for strcat(...) at www.cplusplus.com starts by using strcpy(...) which is one way to get a c string started. Another way would be
s[0] = 0 ;
Also, the parentheses around d[0].tag and d[0].value are harmless but unnecessary.
At the start, the variable s is not a c string. The example for strcat(...) at www.cplusplus.com starts by using strcpy(...) which is one way to get a c string started. Another way would be
s[0] = 0 ;
You don't need to "get a c string started", just declare it large enough to hold the longest string that will be put in it.
Declaring it static or making it global will work the first time. Because I have only seen a snippet of code, who knows what happens the second time through.
