ESP32 PANIC at snprintf

using :

  • Arduino IDE 2.2.1
  • ESP32-S3 DEV1C clone
  • try the 'bytefiles.h' feature with like
    const char* INDEX_HTML = R"literal(_LONG HTML ___)literal";
    and
    #include "bytefiles.h"

when i use
snprintf(DATA, 5000, INDEX_HTML,"FirstStringReplace");
OK, but when i use
snprintf(DATA, 5000, DATA_HTML,"FirstStringReplace");
get:
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.

already try to separate in parts and found 2 of 4 cause that panic?

already did many tests about long content with long replacement of %s ...
adding up near to 5000 ( need max about 3700 ++ dynamic content ) and it worked,

learned that

  • not need to \" ( it is done automatically)
  • also \n are added for each line
  • need %% to print a % ( escape like for html style width)

but no idea what in that particular content in a RAW can be problematic.


more tests found:
a %0.1f in the RAW must be fed by snprintf correctly:

  • no feed PANIC
  • "txt" PANIC
  • 1 PANIC !!!!! << ISSUE >>
  • 1.1 OK

Can you provide a small wokwi / code demonstrating your issue?

Have you tried putting the raw literal in its own .h tab and import it ?

Do you have only one placeholder in the format string?

    • no, have not tested if there is a difference if the literal is imported ( like i do ) or not
    • my example code now imports 6 literals from bytefiles.h in each (15sec) loop well
      ( for 250 loops already ) as long the %f is filled correctly
    • each literal has multiple %

i expect that unfilled % just stay as they are... and NOT lead to panic
looks for me like only the %f has that issues, compiles good, goes to panic at snprintf

my test code is lengthy as i do many tests incl. memory checks..
and document it in the code.

so for show i would need to write a cleaned up version,
just wanted to know if some issues are known?

my workaround will be to replace %f with %s and format floats to string outside to feed.

a shorter example for PANIC

  • even the literal is in the .ino file
  • and the first %f is fed with "txt"

would be

const char* DATA_test = R"literal(
  here we need test escape %% ,
      is that a bad one?                     %.1f ,
                                                  ,
  val1 : %s                                       ,
                                                  ,
  val2 : %s                                       ,
      is that a bad one?                     %.1f ,

)literal";

#define BIG 6000
char DATA[BIG];

void setup() {
  delay(500);
  Serial.begin(115200);
  delay(500);
  Serial.println("test snprintf about %f ");
}

void test_DATA_test_TO_DATA() {
  Serial.printf(" DATA BIG: %d : %d \n",sizeof(DATA),strlen(DATA));
  snprintf(DATA, BIG, DATA_test,"FirstStringReplace");
  Serial.println(DATA);
  Serial.printf(" DATA : %d \n",strlen(DATA)); //
  DATA[0] = '\0'; // EMPTY
 
}

void loop() {
  test_DATA_test_TO_DATA();
  delay(15000);
  // put your main code here, to run repeatedly:
}


/*
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x44c
load:0x403c9700,len:0xbe4
load:0x403cc700,len:0x2a68
entry 0x403c98d4
test snprintf about %f 
 DATA BIG: 6000 : 0 
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x400556d5  PS      : 0x00060c30  A0      : 0x82015ac8  A1      : 0x3fceba90  
A2      : 0x00001770  A3      : 0x0000176c  A4      : 0x000000ff  A5      : 0x0000ff00  
A6      : 0x00ff0000  A7      : 0xff000000  A8      : 0x00000000  A9      : 0x3fc95580  
A10     : 0x3fc9557f  A11     : 0x3fcec89c  A12     : 0x00000001  A13     : 0x3fdfffff  
A14     : 0x7ff00000  A15     : 0x000649c2  SAR     : 0x0000001f  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00001770  LBEG    : 0x400556d5  LEND    : 0x400556e5  LCOUNT  : 0xffffffff  


*/

it's not really what you expect that matters, it's what the code does... :slight_smile:

the spec states

After the format parameter, the function expects at least as many additional arguments as needed for format.

With variadic (variable number of parameters passed to a function), the function code doesn't know how many parameters to expect, so the function will just look into the format string to count the %xxx format placeholders and then tries to access that many parameters that are expected to be on the stack.

if you don't provide enough parameters, the function will kinda interpret what it finds on the stack as the parameters. for example if you have a %s it will expect to find a pointer to a c-string as the next element on the stack, will read the bytes that are supposed to represent a pointer and go read whatever is in memory at that address until the next '\0'. If the address is garbage (pointing at a forbidden area) then you end up in undefined behavior land...

2 Likes

unused "%XXX" in the format string lead to undefined behavior.

3 Likes

snprintf(DATA, BIG, DATA_test,1,"t1","t2",2.0);

still should not lead to PANIC
just as a 1 is not understood as 1.0

I'm oversimplifying but to try to explain further : in C or C++ when you call

myFunction(a,b,c);

the compiler puts the arguments on the stack in reverse order and then calls the function

push c onto the stack 
push b onto the stack
push a onto the stack
call myFunction

the processor at any point in time knows where the top of the stack is and because of type declarations in your function's prototype, the compiler knows how many bytes are needed for a, b, and c. So when you access a in your function, the compiler knows the start address in memory for a is the top of the stack minus the number of bytes needed to represent a. In the same way, to access b, it knows it's the start address of a minus the number of bytes needed to represent b etc... It's all well defined.

When you have a function with a variable number of arguments (we call that a variadic function) the function's prototype does not give any clue to the compiler about the type or number of parameters you are passing. So the compiler cannot really know where the various variables are on the stack as it does not know how many bytes are needed to represent the variables.

the printf() function is such a variadic function. The format string you pass helps the function count how many parameters to expect and thanks to the format specifier (like %d or %f or %s) the function understands what is the size of that parameter (number of bytes to read from the stack).

This is how printf() (and its derivative) can access the right data of the stack.

Say you do something like this:

const char * hour = "22";
char timeBuffer[10];
snprintf(timeBuffer, sizeof buffer, "%s:%s:%s", hour); 

The format string "%s:%s:%s" has two placeholders (in red) for which there is no parameter on the stack.

The function will think you have passed 3 parameters of type pointer to a c-string and use the va_list to traverse the list of parameters and will expect to find 3 pointers on the stack to read from. Unfortunately there is only one and so the function will try to read from the stack whatever garbage there is to read in memory ➜ At that point you no longer honour the specification of the function and anything can happen including panic if you read from a forbidden area in memory.

does it make more sense?

2 Likes

? you want to convince me that i have to live with what i get ?
and my < ISSUE > is my problem, well see here:

there is a warning , that's something
but use a 1 in float as 0.0 is still questionable for me.

I'm not sure what architecture this is compiled for, I assume it's on your PC.

you call

snprintf ( buffer, 
           100, 
           "test two float %0.1f and %0.1f»,
           1,
           2.2 ) ;

so as explained the compiler puts the parameters on the stack in reverse orders. The compiler has rules for variadic functions or integral literals. 1 is seen as an int and 2.2 as a floating point number which will be passed as a double. 100 is part of the function's prototype so the compiler knows its type, it's size_t.

where you get "lucky" it's likely because of compiler padding and memory alignment. the 1 seen as an int will occupy 4 bytes only but the next argument is a double on 8 bytes and the compiler likely aligns on 64 bits and thus adds 4 0 bytes after the int

if you run this code in your simulator

#include <cstdarg>
#include <iomanip>
#include <iostream>

void printBytes(const void* data, size_t size) {
    const unsigned char* bytes = static_cast<const unsigned char*>(data);
    for (size_t i = 0; i < size; ++i) {
        std::cout << std::hex << std::setw(2) << std::setfill('0') << static_cast<int>(bytes[i]) << " ";
    }
    std::cout << std::dec << std::endl;
}

void variadicFunction(char* buffer, size_t bufferSize, const char* format, ...) {
    va_list args;
    va_start(args, format);

    // Assume the 2 double arguments are on the stack even if we passed an int and a double
    double firstDouble = va_arg(args, double);
    double secondDouble = va_arg(args, double);

    // Print the raw bytes of the doubles
    std::cout << "Bytes of first double: ";
    printBytes(&firstDouble, sizeof(double));
    std::cout << "Bytes of second double: ";
    printBytes(&secondDouble, sizeof(double));

    // Print the values of the doubles
    std::cout << "Value of first double: " << firstDouble << std::endl;
    std::cout << "Value of second double: " << secondDouble << std::endl;

    // Clean up the va_list
    va_end(args);
}

int main() {
    char buffer[100];
    variadicFunction(buffer, 100, "test two float %0.1f and %0.1f", 1, 2.2);
        
    // Print the formatted string
    snprintf(buffer, 100, "test two float %0.1f and %0.1f", 1, 2.2);
    std::cout << "Formatted string: " << buffer << std::endl;
        
    return 0;
}

I wrote a variadicFunction which takes the same format as snprintf() and I look into the stack to mimic what snprintf() would do

you'll see in the console

Bytes of first double: 01 00 00 00 00 00 00 00 
Bytes of second double: 9a 99 99 99 99 99 01 40 
Value of first double: 4.94066e-324
Value of second double: 2.2
Formatted string: test two float 0.0 and 2.2

what "saved" you is the padding the first double is seen as "01 00 00 00 00 00 00 00" where the first "01 00 00 00 " is the integer 1 (4 bytes) represented in little endian format and then you had 4 padding bytes which makes the second argument (the real double) start at the expected position in the stack and you read "9a 99 99 99 99 99 01 40" which is the IEEE representation of 2.2

if you look at the "01 00 00 00 00 00 00 00" interpreted as a double though, the value is 4.94066e-324 so when snprintf tries to print it using your format %0.1f it's basically 0. if you modify the snprintf() to ask for exponential notation with %e instead of %0.1f

    snprintf(buffer, 100, "test two float %e and %0.1f", 1, 2.2);

then you'll see in the console

Formatted string: test two float 4.940656e-324 and 2.2

➜ "01 00 00 00 00 00 00 00" has been interpreted as expected


Now this is on a 64 bit architecture. if you were trying to run this on an ESP32 you would get a totally different behavior. The padding might be gone and the way the stack is used be different.

well - that's what undefined behavior means. You are out of the spec so you get weird unpredictable results.

1 Like

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.