How to send a big 2D array to the browser with request on ESP8266 (chunked response)

Hey,
I have a big 2D char array with a lot of data and I need to send it to the browser using request. The microcontroller I use is ESP8266.

My array is:
static char logs[300][100];

And method which I use to send data to browser:

  server.on("/temperature", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(200, "text/plain", sensVal);
  });

And I was thinking about a method to save the data from the array using a for loop to a txt file in SPIFFS memory, and then choose this file to be sent in request. But is this method is good? Or maybe any other better methods exist?

  server.on("/logData", HTTP_GET, [](AsyncWebServerRequest *request){
    request->send(SPIFFS, "/index.html", String(), false, processor);
  });

Anything that works is fine. You need to define what would make it "better".

I totally don't know how to write log array to the browser. Generally I would like to send data to browser without using SPIFFS. And I care about the method that is most efficient and least complicated.

More importantly, what does the browser expect to receive? This is a request, right?

Why are you sending it to browser? For someone to read on screen as text? To download to PC as a text file?

I suggest you either look at the source code for request->send, or read up on how HTTP works. All the server is doing is sending a simple header, followed by the reply text. All as defined by the HTTP protocol.

This page gives a good overview of the response format:

HTTP - Responses (tutorialspoint.com)

I meat that I need to view that data in the browser. The logs are something like Serial.print but it is only for me to see. In browser I need to have it as a plain text. Like this (300 rows):

Yes, for someone to read the text on the screen :slight_smile:
I mean, someone open the browser, enter the URL like "192.168.0.6/logging" and gets all the data from all rows from big array.

I had the logs in one big String previously but it was very bad because it caused fragmentation in heap memory :slight_smile:

Yea, I know how HTTP works. But I have a problem to send a one big array. I mean that I can't use a for loop to generate the big String file with all array rows because by generating a temporary String file the microcontroller will be out of heap memory.

I'm guessing a bit here....

server.on("/logData", HTTP_GET, [](AsyncWebServerRequest *request){
  AsyncResponseStream *response = request->beginResponseStream("text/html");
  for (int l=0; l<300; l++) response->print(logs[l]);
  request->send(response);
  });

Wow, it looks really good and easy. I used it to my code but it not works so good. I mean that not every row from array are printed. I don't know why.
The array is full of 300 rows with text but in browser I get only 41 and penultimate line in browser is empty too. Do you know why maybe?

My code:

server.on("/logs", HTTP_GET, [](AsyncWebServerRequest *request){
    AsyncResponseStream *response = request->beginResponseStream("text/plain; charset=UTF-8");
    for (int z = 0; z < 300; z++){
      response->print(logs[z]);
      response->print("\n");
    }
    request->send(response);
  });

Btw with "text/html" I have this same results but without new line every row.

Perhaps the above should be:

  for (int l=0; l<300; l++) {
     response->print(logs[l]);
     request->send(response);
    }

With this:

server.on("/logs", HTTP_GET, [](AsyncWebServerRequest *request){
    AsyncResponseStream *response = request->beginResponseStream("text/plain; charset=UTF-8");
    for (int z = 0; z < 300; z++){
      response->print(logs[z]);
      response->print("\n");
      request->send(response);
    }
  });

I get:

Btw my free memory Heap is only 10KB with this big array.

Reduce your logs[] array to 30 lines. Does that make any difference? If it works correctly at 30 lines but not 300, this could mean there is a problem with insufficient memory.

Are the text lines in logs[] null-terminated? Only the first part of each line "46 |" seems to appear correctly. What should the rest of that line look like, for example?

How did you produce the example in post #7?

I reduced this to 200 lines in array and now I can get 74 lines in browser :confused:
When reduce to 170 I get 83 rows.
With 130 lines in array I get 99 in browser.
Not good :confused: Is it sounds like this method doesn't work?

I think that all rows in array are null-terminated.
I did this by that code:

int logNumber = 0;
const int iloscLogow = 200;
const int dlugoscJednegoLogu = 100;
static char logs[iloscLogow][dlugoscJednegoLogu];

void logThis(String str) {
  str = String(random(1000, 9999)) + "_AęąźńćąąąAAAAAAAAWWWWWWWWWWWWWTTTTTTTTTTTTTTIIIIIIIIIIFFFFFFFFFFFFFPPPPPPPPSSSSSSSS_______100";
  logNumber++;
  String tempStr = String(logNumber) + "| " + str;
  for (int x = iloscLogow - 1; x > 0; x--){
    strcpy(logs[x], logs[x - 1]);
  }
  tempStr.toCharArray(logs[0], dlugoscJednegoLogu);
}

The one row should looks like:

43| 7492_AęąźńćąąąAAAAAAAAWWWWWWWWWWWWWTTTTTTTTTTTTTTIIIIIIIIIIFFFFFFFFFFFFFPPPPPPPPSSSSSSS

The next:

42| 6464_AęąźńćąąąAAAAAAAAWWWWWWWWWWWWWTTTTTTTTTTTTTTIIIIIIIIIIFFFFFFFFFFFFFPPPPPPPPSSSSSSS

Btw. I am sure that all data in logs are visible because I can print all 300 rows using Serial.println.

I produced it with old method of this what I am doing now but it was very bad because it used the String appending and this caused big memory fragmentation.
The old script is:

void logThis(String str) {
  logNumber++;
  logs = logs + logNumber + "| " + str + "\n";
}

Sorry, I have no other suggestions. It seems to be an insufficient memory problem, but I guess the solution I suggested used as much memory as the other methods you tried.

I hope you find an answer.

Not good :confused: I dont know who could help me with this. If official arduino forum don't know it will be very hard to find other idea.

This is not an Arduino problem. It is a communications problem between two completely different platforms.

Yea, I know, I mean that there are a lot of specialist guys and it will be harder to find a helpful ideal to resolve it in other forums than this.

My suggestion would be to search this forum for topics related to "AsyncWebServerRequest" and similar terms. Which forum members seem to be most familiar and most expert with the library?

Ping them on this topic (with @...) and ask if they can help you.