Fread() function stops copying data into buffer when encountering null bytes

I'm having a strange issue when using the Arduino Giga R1. I'm trying to make a webserver with it, which I've got almost working completely, but if I have any images on the page, the fread() function I'm using to read files from a flash drive seems to read the file, but will stop copying the file into the buffer once it hits the first null byte. I have been able to confirm this by analyzing the original file, and the file I get in my browser using a hex editor. I've tried using fgets() instead, but that function seems to completely fail after reading just a few bytes. I set up the server so that it can serve pretty much any file and notify the browser of the file's type and size. I should also mention, I'm using the Giga's SDRAM chip to act as the buffer. It's 8MB, and the image in question here is only about 2.7MB, so memory capacity isn't the issue here. I have also tried this with different, smaller images, and I'm getting the exact same problem with those, as well. The entire webpage loads just fine (html, js, css files all load correctly), it's only the images that will not load properly. As a possible workaround, I've thought about converting images to base64 before storing them into SDRAM and then transmitting them to the client, but I haven't figured out how to do that yet. I don't even know if that's a viable option. As a side note, I'm starting to believe the entire problem might be related to the Giga's MbedOS. I think the MbedOS implementation on the Giga might be screwing around with read/write functions and erroring out when encountering null bytes.

Here's the code in question:

FILE *f = fopen(filename.c_str(), "rb");

      if (f) {
        client.println(F("HTTP/1.1 200 OK"));
        //client.println(F("Connection: close"));
        client.print(F("Content-Type: "));
        const char* ext = filename.substring(filename.indexOf(".")).c_str();
        Serial.print(F("Extension: "));
        Serial.println(ext);
        String contentType = getContentType(ext);
        Serial.println(contentType);
        client.println(contentType);
        client.print(F("Content-Length: "));
        fseek(f, 0, SEEK_END);
        long filesize = ftell(f);
        rewind(f);
        Serial.print("File size: ");
        Serial.println(filesize);
        client.println(filesize);
        client.println();
        
        char *buffer;
        buffer = (char*)SDRAM.malloc(sizeof(char) * (filesize + 1));
        //memset(buffer, 'F', sizeof(char) * (filesize + 1));
        size_t bytesRead = fread(buffer, sizeof(char), filesize, f);
        Serial.print("Bytes read: ");
        Serial.println(bytesRead);
        Serial.print("buffer size: ");
        Serial.println(strlen(buffer));
        for (int i = 0; i < strlen(buffer); i++) {
          Serial.print(buffer[i], HEX);
        }
        Serial.println();
        client.write(buffer);
        client.flush();
        fclose(f);
        SDRAM.free(buffer);
      }

I've also uploaded two images showing the raw data of the original image and the image my browser gets from the Arduino in the hex editor. I have also uploaded a screenshot of the Serial output showing that fread() did in fact read the entire image file, but the final size of the buffer is only 8 bytes, which corresponds to the 8 bytes of the image file just before encountering the first null bytes in the original image.

So, what's going on here? Why does fread() stop copying the image data into the buffer once it encounters the first null byte in the image?



Your program is working properly, as you have programmed it. "A string is a special array that has one extra element at the end of the string, which always has the value of 0 (zero). This is known as a "null terminated string".". When you ask for strlen(), you are getting the length of the array, up to the first null character. All is working as you programmed it.

Well that explains why strlen() only shows 8 bytes (and thus why the Serial.print() is only printing out the first 8 bytes of the buffer), but it does NOT explain why my web browser is only receiving the first 8 bytes of the image. That’s what the first and second images are showing. The first image is the raw data of the original image, and the second image is the raw data received by my browser from the Arduino. The strlen() function here is purely for debugging purposes. It has nothing to do with transmitting the image data to the client and why that image data is incomplete. When I have time, I’ll change that for loop to this and see what happens:

for (int i = 0; i < filesize; i++) {
    Serial.print(buffer[i], HEX);
}

I realized I haven’t entirely narrowed down which function is causing my problem.

Why do you think that?
Buffer sizes on the picture above are calculated by strlen() function too and obviously wrong. See the real bytes count in "Bytes read" line of the output.

I do not think uniplast21 actually wrote the program.

The strlen() function is only being used to print the contents of the buffer. That’s it! Nothing more! Do you not see that the for loop where strlen() is being used only has a Serial.print() function. Serial.print only prints to the console. It has nothing to do with transmitting data to a client. The fread() function is the one filling the buffer with data, and the client.write() function is transmitting the data in the buffer to the web browser. It’s not at all difficult to see that.

If requested to print a C-string (a zero terminated character array), Serial.print stops at the zero terminator. In other words, it uses strlen() to determine how many characters to print.

Avoid using String objects, and your life will be less complicated and confusing.

Instead of

   client.write(buffer);

use the form that provides len, the correct number of bytes to be written.

   client.write(buffer, len);

Aha!! Thank you! Using client.write(buffer, filesize); fixed my problem!

It is because your buffer has a char* type. Passing it to client.write(buffer) as pointer to char treat it as a null terminated string too!

Wait, so does client.write() use the strlen() function behind the scenes to determine how much of the buffer to send?

No.
The write() is overloaded function. The version write( char* b) treat the input as null terminated buffer, therefore it send the data until the first zero byte.

Ohhhh ok I completely misunderstood you earlier. I apologize! Thank you for clarifying!

my English is not good, sorry

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.