I have seen a few arduino entires about accessing an SD Card from an Arduino Web Server. Not sure if anyone has got that working yet with the PortentaH7 and the Vision shield SD card over WiFi?
Any suggestions?
Based on my projects I have:
- I have my Web Server (HTTPD server) running, but responding the requested pages out from MCU memory, not (yet) from SD Card
- I have SD Card running, to read/write files on SD Card, into any MCU memory
it should be a "piece of cake" to read and respond the web page from SD card:
read response (file) from SD Card, place into buffer and let respond with this buffer content.
There are only two concerns I would raise as questions:
a) is reading a file as HTTPD response from SD Card fast enough? The SD Card speed matters.
Worst case: the HTTPD request for a page times out because SD Card is too slow. The SD Card cannot provide the HTTPD response right on time (or the INTs block each other).
b) When sending back the HTTPD response (the page) - should it be a single TCP packet, so
that entire page read from SD Card sits in buffer for a single packet HTTPD response?
I think so: I assume, the entire page must be sent with a single TCP HTTPD response, not sent
in TCP chunks (file split into pieces).
This means: do you have enough memory to read an entire page from SD Card and afterwards you send this entire buffer as TCP HTTP response?
I am pretty sure, it will work (if you have HTTPD server running, SD access working). Just a question for response time, FW latency (speed) and internal temporary buffers needed.
Maybe you have to limit the size of max. page size for a respond (for a page requested).
But technically - I do not see any issue to do so.
1 Like
BTW:
The response has a specific format, e.g. a header with HTTP version etc.
You can store this info in SD Card response file, or: you had to generate the HTTPD response (head and tail) and place the "text content" in between, from SD Card.
Give it a try:
when you have a HTTPD server, reading pages from memory, now read content from a SD Card file, send this buffer as response ... just check the timing (delay caused by SD Card operation and response delay accepted by client - but should be in seconds range), and temporary buffer sized needed to read from SD Card and send as TCP response (MCU memory size limitations).
1 Like
YEs @tjaekel the response time is the thing I have been working on. When I send a PNG from the Portenta directly to a webpage served by the protenta, small images work fine say 96x96 but as the image gets larger the webpage times out that is a pain.
So I got websockets working with the WiFiSSLClient code here and the page now does not timeout, but the processing of the "mask" for the PNG to be sent by a websocket is really long and a real pain. Suppossedly you can send a websocket without the 4 byte "mask" but it does not seem to work with modern websockets. Such a pain.
So what is working for me is just to have the fast websocket tell the Portenta when to save the PNG image to SD card (I make machine learning models using edgeimpulse.com and accurate images are very important.) example code here
With the latest method if I could access the sd card images from a local web that would be a fine solution. If anyone has Portenta sd card to webpage code I would love to see it. By the way these images are only 320 x 320 so from a web point of view they are not very big
Some good code here for the sd card to webpage video: Arduino Web Server with Webpage Stored on SD Card - YouTube
code here:
relevant sketch here
//------------------------------------------------
//Arduino Web Server via Ethernet Shield & SD Card
//------------------------------------------------
#include <SPI.h>
#include <Ethernet.h>
#include <SD.h>
//------------------------------------------------
byte mac[] = {0x90, 0xA2, 0xDA, 0x00, 0x4A, 0xE0};
EthernetServer server(80);
//------------------------------------------------
File HMTL_file;
//=================================================================
void setup()
{
Serial.begin(9600);
//---------------------------------------------------------
Serial.println("Initializing SD card...");
if(!SD.begin(4))
{
Serial.println("SD card initialization failed!");
return;
}
Serial.println("SD card initialized.");
//---------------------------------------------------------
if(!SD.exists("webpage.htm"))
{
Serial.println("webpage.htm file not found!");
return;
}
Serial.println("webpage.htm file found");
//---------------------------------------------------------
Ethernet.begin(mac);
server.begin();
Serial.print("Server Started...\nLocal IP: ");
Serial.println(Ethernet.localIP());
}
//=================================================================
void loop()
{
EthernetClient client = server.available();
if(client)
{
boolean currentLineIsBlank = true;
while(client.connected())
{
if(client.available())
{
char c = client.read();
if(c == '\n' && currentLineIsBlank)
{
//send HTTP response header to client
client.println("HTTP/1.1 200 OK\n\rContent-Type: text/html\n\r\n\r");
//-------------------------------------------------------------------
//send HTTP file to client
HMTL_file = SD.open("webpage.htm");
if(HMTL_file)
{
while(HMTL_file.available()) client.write(HMTL_file.read());
HMTL_file.close();
}
break;
}
//---------------------------------------------------------------------
if (c == '\n') currentLineIsBlank = true;
else if(c != '\r') currentLineIsBlank = false;
}
}
delay(10);
client.stop();
}
}
I will see what I can do with that.
Here is another possibly useful link
Kind of got everything I wanted working, but the code is very messy. I will tidy it up but for anyone interested here is the code so far:
/*
WiFi Web Server LED Blink
Enter you wifi and password in the code below
A simple web server that lets you blink an LED via the web.
This sketch will print the IP address of your WiFi module (once connected)
to the Serial monitor. From there, you can open that address in a web browser
to turn on and off the LED on pin 9.
If the IP address of your board is yourAddress:
http://yourAddress/H turns the LED on
http://yourAddress/L turns it off
This example is written for a network using WPA encryption. For
WEP or WPA, change the Wifi.begin() call accordingly.
created 25 Nov 2012
by Tom Igoe
*/
#include <Arduino.h> // Only needed by https://platformio.org/
#include "WiFi.h"
// for SD card
#include "SDMMCBlockDevice.h"
#include "FATFileSystem.h"
SDMMCBlockDevice block_device;
mbed::FATFileSystem fs("fs");
int mode = 0;
// Choose either the following arduino_secrets.h tab and bracket out the next 2 lines after it
// That route is more secure.
// Or just change the lines below for your Network and Password. Eaier but not as secure
// Since if you share this file it will have your info on it
//#include "arduino_secrets.h" // more safe
#define SECRET_SSID ""
#define SECRET_PASS ""
char ssid[] = SECRET_SSID; // Changing the secret strings to a character array
char pass[] = SECRET_PASS;
int keyIndex = 0;
int status = WL_IDLE_STATUS;
WiFiServer server(80);
void printWifiStatus() {
// print the SSID of the network you're attached to:
Serial.print("SSID: ");
Serial.println(WiFi.SSID());
// print your board's IP address:
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);
// print the received signal strength:
long rssi = WiFi.RSSI();
Serial.print("signal strength (RSSI):");
Serial.print(rssi);
Serial.println(" dBm");
// print where to go in a browser:
Serial.print("To see this page in action, open a browser to http://");
Serial.println(ip);
}
void setup() {
Serial.begin(115200); // initialize serial communication
delay(5000);
Serial.println("Wait a bit to connect serial monitor");
delay(5000);
Serial.println("Wait a bit");
delay(5000);
Serial.println("Wait a bit");
pinMode(LED_BUILTIN, OUTPUT); // set the LED pin mode
pinMode(LEDB, OUTPUT); // set the LED pin mode
digitalWrite(LEDB, LOW);
// attempt to connect to Wifi network:
while (status != WL_CONNECTED) {
Serial.print("Attempting to connect to Network named: ");
Serial.println(ssid); // print the network name (SSID);
// Connect to WPA/WPA2 network. Change this line if using open or WEP network:
status = WiFi.begin(ssid, pass);
// wait 5 seconds for connection:
delay(5000);
}
server.begin(); // start the web server on port 80
printWifiStatus(); // you're connected now, so print out the status
digitalWrite(LEDB, HIGH);
digitalWrite(LED_BUILTIN, LOW);
Serial.println("Mounting SDCARD...");
int err = fs.mount(&block_device);
if (err) {
Serial.println("No SD Card filesystem found, please check SD Card on computer and manually format if needed.");
// int err = fs.reformat(&block_device); // seriously don't want to format your good data
}
mkdir("fs/myFolder1",0555);
char myFileName[] = "fs/myFolder1/index.html"; // "fs/" needs to be there, think fileSystem
unsigned char c;
FILE *fp = fopen(myFileName, "r"); // "r" read only
while (!feof(fp)){ // while not end of file
c=fgetc(fp); // get a character/byte from the file
//printf("Read from file %02x\n\r",c); // and show it in hex format
Serial.print((char)c); // show it as a text character
}
fclose(fp);
Serial.println("------------------------- Done Showing file --------------------------------");
delay(10000); // wait a bit
}
void loop() {
WiFiClient client = server.available(); // listen for incoming clients
if (client) { // if you get a client,
int x1 = analogRead(A1); // read A1
Serial.println("new client"); // print a message out the serial port
String currentLine = ""; // make a String to hold incoming data from the client
while (client.connected()) { // loop while the client's connected
if (client.available()) { // if there's bytes to read from the client,
char c = client.read(); // read a byte, then
Serial.write(c); // print it out the serial monitor
if (c == '\n') { // if the byte is a newline character
// if the current line is blank, you got two newline characters in a row.
// that's the end of the client HTTP request, so send a response:
if (currentLine.length() == 0) {
// HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
// and a content-type so the client knows what's coming, then a blank line:
client.println("HTTP/1.1 200 OK");
if (mode == 2 ){
client.println("Content-Type: image/png");
client.println();
mkdir("fs/myFolder1",0555);
char myFileName[] = "fs/myFolder1/img01.png"; // "fs/" needs to be there, think fileSystem
char buffer[128];
FILE *fp = fopen(myFileName, "rb"); // "r" read only
//while(fgets(buffer, 128, fp)) { // "r" read only
while(fread(buffer, 1, 128, fp)) {
client.write((uint8_t*)buffer, 128);
// wait(1);
delayMicroseconds(200);
}
fclose(fp);
Serial.println("------------------------- Done Showing file --------------------------------");
delay(1);
client.stop();
currentLine="";
break;
}
client.println("Content-type:text/html");
//client.println("{answer:42}");
client.println();
// the content of the HTTP response follows the header:
client.print("<input type=button value='LED_BUILTIN Off' onclick='{location=\"/H\"}'>");
client.print("<input type=button value='LED_BUILTIN On' onclick='{location=\"/L\"}'>");
client.print("<input type=button value='PRINT FROM SD CARD' onclick='{location=\"/SD-card\"}'>");
client.print("<input type=button value='Load Image img01.png' onclick='{location=\"/image\"}'>");
client.print("<br> AnalogRead(A1); = "+ String(x1) );
if (mode == 1 ){
mkdir("fs/myFolder1",0555);
char myFileName[] = "fs/myFolder1/index.html"; // "fs/" needs to be there, think fileSystem
unsigned char c;
FILE *fp = fopen(myFileName, "r"); // "r" read only
while (!feof(fp)){ // while not end of file
c=fgetc(fp); // get a character/byte from the file
//printf("Read from file %02x\n\r",c); // and show it in hex format
Serial.print((char)c); // show it as a text character
client.print((char)c); // print to web page
}
fclose(fp);
Serial.println("------------------------- Done Showing file --------------------------------");
}
// The HTTP response ends with another blank line:
client.println();
// break out of the while loop:
break;
}else { // if you got a newline, then clear currentLine:
currentLine = "";
}
} else if (c != '\r') { // if you got anything else but a carriage return character,
currentLine += c; // add it to the end of the currentLine
}
// Check to see if the client request was "GET /H" or "GET /L":
if (currentLine.endsWith("GET /H")) {
digitalWrite(LED_BUILTIN, HIGH); // GET /H turns the LED on
}
if (currentLine.endsWith("GET /L")) {
digitalWrite(LED_BUILTIN, LOW); // GET /L turns the LED off
}
if (currentLine.endsWith("GET /SD-card")) { // show SD card
mode=1;
}
if (currentLine.endsWith("GET /image")) { // show SD card
mode=2;
}
}
}
// close the connection:
client.stop();
Serial.println("client disonnected");
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);
Serial.println("");
}
}
Cool! Well done!
I do often the same approach: "hack the code" - get it to work. And later I clean up (e.g. use macros, separate code in files ...).
As long it is working - you did great!
Thanks @tjaekel I did a fair bit of digging, however chatGPT gets credit for the final step. I gave it my hacked together code and it produced this output. Generally I have not had much success with ChatGPT and MBED code it always seems to have 1 or 2 lines that I can't quite fix, but this time it was really useful.
I gave it this. I didn't even see the JPEG instead of PNG messup.
```if (mode == 2 ){
client.println("Content-Type: image/jpeg");
client.println();
mkdir("fs/myFolder1",0555);
char myFileName[] = "fs/myFolder1/img01.png"; // "fs/" needs to be there, think fileSystem
char buffer[128];
FILE *fp = fopen(myFileName, "r"); // "r" read only
while(fgets(buffer, 128, fp)) {
client.print(buffer);
// wait(1);
delayMicroseconds(200);
}
fclose(fp); ``` fix this code to serve a PNG image to a webpage
After testing I found out the delayMicroseconds was not needed.
So I have updated my library the "Portenta Pro Community Solutions" version 0.9.3 to have the latest sd card to web files. To find it on the Arduino IDE just search for "community"
Of interest is the new section "dot36" has all my latest sd card PNG webpage websockets code. The folder is here online
I might add a few more examples if I can figure out how to make a simple webpage sd card server and webpage PNG server.