Arduino Web Server - HTTP requests/Digest Authentication

Hi!

Which is the best way ("the best code") to distinguish between HTTP requests once we have stored them into a buffer?

My code (HTTPRequest is the buffer):

if (strstr(HTTPRequest, "GET / ") || strstr(HTTPRequest, "GET /home.html")) {

    client.println(F("HTTP/1.1 200 OK"));										
    client.println(F("Content-Type: text/html"));
    client.println(F("Cache-Control: max-age=86400"));
    client.println(F("Connnection: close"));
    client.println();

    file = SD.open("home.htm");

    if (!file) {

    	Serial.println(F("No es podrà enviar l'arxiu home.htm al client."));

    }

} else if (strstr(HTTPRequest, "GET /actual.html")) {

    client.println(F("HTTP/1.1 200 OK"));
    client.println(F("Content-Type: text/html"));
    client.println(F("Cache-Control: max-age=86400"));
    client.println(F("Connnection: close"));
    client.println();

    file = SD.open("actual.htm");

    if (!file) {

    	Serial.println(F("No es podrà enviar l'arxiu actual.htm al client."));

    }

} else if (strstr(HTTPRequest, "GET /historic.html")) {

    client.println(F("HTTP/1.1 200 OK"));
    client.println(F("Content-Type: text/html"));
    client.println(F("Cache-Control: max-age=86400"));
    client.println(F("Connnection: close"));
    client.println();

    file = SD.open("historic.htm");

    if (!file) {

    	Serial.println(F("No es podrà enviar l'arxiu historic.htm al client."));

    }

} else if (strstr(HTTPRequest, "GET /registre.html")) {

    client.println(F("HTTP/1.1 200 OK"));
    client.println(F("Content-Type: text/html"));
    client.println(F("Cache-Control: max-age=86400"));
    client.println(F("Connnection: close"));
    client.println();

    file = SD.open("registre.htm");

    if (!file) {

        Serial.println(F("No es podrà enviar l'arxiu registre.htm al client."));

}

} else if (strstr(HTTPRequest, "GET /canvasjs.js")) {

    client.println(F("HTTP/1.1 200 OK"));
    client.println(F("Content-Type: text/js"));
    client.println(F("Cache-Control: max-age=86400"));
    client.println(F("Connnection: close"));
    client.println();

    file = SD.open("canvasjs.js");

    if (!file) {

    	Serial.println(F("No es podrà enviar l'arxiu canvasjs.js al client."));

    }

} else if (strstr(HTTPRequest, "GET /favicon.ico")) {

    file = SD.open("favicon.ico");

    if (file) {

        client.println(F("HTTP/1.1 200 OK"));
        client.println(F("Cache-Control: max-age=86400"));
        client.println();

    } else {

    	Serial.println(F("No es podrà enviar l'arxiu favicon.ico al client."));

    }

} else if (strstr(HTTPRequest, "GET /logouib.png")) {

    file = SD.open("logouib.png");

    if (file) { // Si s'ha pogut obrir l'arxiu 'logouib.png'.

        client.println(F("HTTP/1.1 200 OK"));
        client.println(F("Cache-Control: max-age=86400"));
        client.println();

    } else {

    	Serial.println(F("No es podrà enviar l'arxiu logouib.png al client."));

    }

} else if (strstr(HTTPRequest, "GET /curls.png")) {

    file = SD.open("curls.png");

    if (file) {

        client.println(F("HTTP/1.1 200 OK"));
        client.println(F("Cache-Control: max-age=86400"));
        client.println();

    } else {

    	Serial.println(F("No es podrà enviar l'arxiu curls.png al client."));

    }

} else if (strstr(HTTPRequest, "ajaxInformationONDevices")) {

    client.println(F("HTTP/1.1 200 OK"));
    client.println(F("Content-Type: text/xml"));
    client.println(F("Connection: keep-alive"));
    client.println();

    responseAjaxONDev();

}

I don't like this solution because I have to make an else if statement for each HTTP request. Is there any way to implement it with regexp or something like and obtain a better code?

TY so much!

There is nothing bad with a lot of else if. The only alternative to else if is switch case but your test cases are too complicated to use it anyway.

An alternative to your approach is:

  1. Extract the GET command from HTTPRequest with strcpy()
  2. Compare it with strcmp() and execute your code

Though there are many duplicated codes inside your ifs, take care of that too.

I don't understand your alternative. In the end, it's the same that I'm doing no? Can you make a short pseudocode please?

TY

Kane12:
I don't understand your alternative. In the end, it's the same that I'm doing no? Can you make a short pseudocode please?

TY

Yes, it's the same. As I said your code is "best" already because there is no alternative to the else if

There is no "best" way to do anything.

However,

Perhaps the best way might be to actually parse the HTTP request line and headers into their parts. The HTTP request line has a specific format defined in RFC2616.

METHOD URL VERSION

Method is GET, POST, DELETE, OPTIONS, HEAD, any one of several things.

URL is a percent-encoded URL.

Version is HTTP/1.1 . Technically, you should handle HTTP/1.0, but no-one does and no web clients send that style of request anymore.

Having done that, you can parse the URL into its parts. Usually there is no scheme, host, port, or authority - just the path and the query string. The path is usually the thing that you are interested in.

If you want the query string, then it is WWW-Form-Encoding encoded. This is not the same as percent encoding. First you do the URL decoding to get the query string, then you parse the string to get its keys and values, then you do the www-form decoding on the values. A lot of people get this wrong by not understanding that these are separate processes that must be done in the right order. Actually - I'm not 100% certain if the parsing out of keys and values is part of querystring syntax or part of form-encoding syntax for values. That is, I don't know if form keys might be www-form encoded.

Then parse each header, which look like

key: value

You will want to pay particular attention to the Content-Length header, if it is present, and you may want Content-Type, too.

After the blank line indicating the end of the headers, you need to deal with the content. GET requests have no content. POST requests do. If you want to leave the pipe open and the client supports Connection: Keep-Alive, then content in the pipe needs to be consumed. To do this, you read out of the pipe the number of bytes specified by the Content-Length header. If there is no Content-Length, then you can read bytes until there are no more, or you can just close the pipe prematurely.

What you do with those bytes depends on what payload you are handling. If it is a posted HTML form, the content will be WWW-Form-Encoded . If it is JSON, then - well - it will be JSON. But it might be any one of several things. If it's something you are not prepared to deal with, there are appropriate HTTP response error codes to send in reply.

Is suppose the real answer to your actual question, what is the best way, is: "HTTP is somewhat complicated. Get hold of a library that does what you need, and use that".

I also struggled to put in switch case and found no efficient alternative to else if.

in your previous question a gave you a link to my arduino WebServer. study it. I am a professional coder with 25 years experience.

You're right! @Juraj

ChunkedPrint chunked(client, buff, sizeof(buff));

     if (l == 2 && strchr("IELAPHVS", fn[1])) {
          webServerRestRequest(fn[1], chunked);
     } else {
          webServerServeFile(fn, chunked);
}

Can you explain this code please?

Kane12:
You're right! @Juraj

ChunkedPrint chunked(client, buff, sizeof(buff));

if (l == 2 && strchr("IELAPHVS", fn[1])) {
          webServerRestRequest(fn[1], chunked);
    } else {
          webServerServeFile(fn, chunked);
}




Can you explain this code please?

I send Ajax REST requests for json data in the form of one character. example http://192.168.1.6/E for events. if the length of the request is 2 like "/E" and the second character is one of the list in strchr then we serve json data. Otherwise the request is for a file from file system so we serve it.

ChunkedPrint is a wrapper implementation of Print with buffering and an option to output the body of the request in chunked encoding transport format, which can be used if we do not know the size of the response body when we build the header of the response.

But in the case of sending files, why do you use it if you know the length of the body with dataFile.size(). Which is the benefit of use ChunkedPrint? I say it because if the browser knows the length he knows when he can close the connection with the server (he doesn't need the ChunkedPrint strategy to know when he has received all the data).

On the other hand, in my case I fill a buffer with the information that I want to send and then I do a client.print() of this buffer (it seems like it's much faster send a lof of bytes instead of send a unique byte when you do a client.print()). Suposing that using ChunkedPrint's strategy is better that not using it, can I follow this strategy and use the CStringBuilder as you say in the document?

PS1: your code is awesome, can I take it and adapt it to my project?

PS2: I have been investigating. Here's an example of encoding and decoding of ChunkedPrint's strategy:

4\r\n
Wiki\r\n
5\r\n
pedia\r\n
E\r\n
in\r\n
\r\n
chunks.\r\n
0\r\n
\r\n

Wikipedia in

chunks.

In my case, I have to send more or less 5-10 MB to the client. Do you think this strategy is appropiate in my case (I think not, but I want to know your opinion)? I have tested the transmission speed of the data and it's about 30 KB/s...

Kane12:
But in the case of sending files, why do you use it if you know the length of the body with dataFile.size()?
Why do you don't use CStringBuilder?

I create ChunkedPrint because I need chunked encoding in the json response. In the files serve function I don't activate the chunked encoding, but the buffering is used. The json responses wouldn't fit into memory when the project was on Uno.

Kane12:
For other hand, in my case I fill a buffer with the information that I want to send and then I do a client.print() of this buffer (it seems like it's much faster send a lof of bytes instead of send a unique byte when you do a client.print()). Can I follow this strategy and use the CStringBuilder?

CStringBuilder will help you to fill the buffer you used. With BufferedPrint or ChunkedPrint the output is buffered so you can print directly and the data are not send in small peaces.

Kane12:
PS: your code is awesome, can I take it and adapt it to my project?

of course. StreamLib is in Library Manager, you can install it in IDE. And I published the Regulator project for sharing.

I have edited my reply, please read it!

Kane12:
PS2: I have been investigating. Here's an example of encoding and decoding of ChunkedPrint's strategy:

4\r\n

Wiki\r\n
5\r\n
pedia\r\n
E\r\n
in\r\n
\r\n
chunks.\r\n
0\r\n
\r\n

Wikipedia in

chunks.




In my case, I have to send more or less 5-10 MB to the client. Do you think this strategy is appropiate in my case (I think not, but I want to know your opinion)? I have tested the transmission speed of the data and it's about 30 KB/s...

the idea of chunked transfer is to send continuous data. I assume you use a Mega, because Ethernet + SD leave very little RAM on Uno. with 4k buffer for BufferedPrint or ChunkedPrint I think it is feasible to stream megabytes of data.

Yeah, I'm using a Mega. But if you see the example that I have put, this strategy increases the number of bytes that have to be sent no?

Wikipedia in

chunks.

It has more or less 20 bytes.

4\r\n
Wiki\r\n
5\r\n
pedia\r\n
E\r\n
in\r\n
\r\n
chunks.\r\n
0\r\n
\r\n

It has more or less 70 bytes?

The increase of the number of bytes to be sent is high. If I have to send 5 MB of real data and I apply this strategy, how big will the body of the HTTP response be? And we have to consider that the data transmission speed is about 30 KB/s...

PS: now I understand what are you saying. I don't have to apply this strategy. I only have to use buffering. But I have the following question: if I don't need to apply this strategy, which is the difference between use client.write(buffer, length) or use the following?

ChunkedPrint chunked(client, buff, sizeof(buff));

while (dataFile.available()) {
       bp.write(dataFile.read());
}

Kane12:
The increase of the number of bytes to be sent is high. If I have to send 5 MB of real data and I apply this strategy, how big will the body of the HTTP response be? And we have to consider that the data transmission speed is about 30 KB/s...

If you know the size, set it in http header. if you don't know it, you can send the data without size information and then close the connection. after some seconds the browser evaluates it as the end.

with chunked encoding for 4000 kB with 4kB buffer it would be 5 kB more data. Every chunk size line would be 5 bytes (FA0\r\n)

with chunked encoding for 4000 kB with 4kB buffer it would be 5 kB more data. Every chunk size line would be 5 bytes (FA0\r\n)

Ok! But... as I know with client.print() you only can print a buffer or more or less 2900 B. Using your library, can I use a 4KB buffer? Or this it only depends of the print() function?

Kane12:
Ok! But... as I know with client.print() you only can print a buffer or more or less 2900 B. Using your library, can I use a 4KB buffer? Or this it only depends of the print() function?

I did a test now, with modified ChunkedPrintExample.ino, with 4000 bytes buffer, long l and i in printSomeData() and more than 50000 bytes output. it works

Ok @Juraj. Thank you very much!!!

PS: since you have a lot of experience in programming, is a good practice to use sprintf() function in cases like this:

sprintf(dateActualTemp[indexDevicesSampled[i]], "%04d/%02d/%02d", year, month, day);

Or exists another function that is better in efficiency?

Kane12:
Ok @Juraj. Thank you very much!!!

PS: since you have a lot of experience in programming, is a good practice to use sprintf() function in cases like this:

sprintf(dateActualTemp[indexDevicesSampled[i]], "%04d/%02d/%02d", year, month, day);

Or exists another function that is better in efficiency?

printf functions take some program memory. but I used them and I still had 1 kB flash free at the end with Regulator project Uno version (without Blynk and SD). on Mega you have enough program memory.

bp.printf(F("Expires: %s, "), dayShortStr(weekday(expires)));

Instead of using Expires you can use Cache-control: "time in seconds". It's the same and you can delete those two printf instructions.

I have a question for you: if I have to use a variable (for example File file) in many parts of the code (even in many functions), speaking in memory terms, it's better to declare it as global or declare it when it has to be used?

An example:

File file;

void setup() {
 file = SD.open();
 // some stuff
}

void loop() {
 file = SD.open();
 // some stuff
 doThings();
}

void doThings() {
 file = SD.open();
 // some stuff
}

In this example we have always one file variable.

Another example:

void setup() {
 File file = SD.open();
 // some stuff
}

--> Ok, when setup finish file is deleted from memory but...

void loop() {
 File file = SD.open();
 // some stuff
 doThings();
}

void doThings() {
 File file = SD.open();
 void doThings2();
 // some stuff
}

void doThings2() {
 File file = SD.open();
 // some stuff
}

... when we execute doThings2() function, in memory we have 3 file variables no? Because loop and doThings have not finished yet.

And now... which is the best solution in cases likes this (speaking in memory terms. I wanna obtain a code with the minimum consume of SRAM possible).