Saving data from an unsigned char* variable to a char*

[filipekerplunk]

Helo!

I'm getting a message via mqtt of type unsigned char* and I'm trying to pass this data to a char[], but when running my ESP32 restarts and enters a loop.

What could I be doing wrong?

void callback(char* topic,  unsigned char* message, unsigned int length) {
  dbSerialPrint("Mensagem chegou no topico: ");
  dbSerialPrint(topic);
  dbSerialPrint(". Tamanho: ");
  dbSerialPrint(length);
  
  char messageTemp[length];
  for (int i = 0; i < length; i++) {
    //dbSerialPrint((char)message[i]);
    messageTemp[i] = (char)message[i];
  }
}

Thanks!

[ToddL1962]

I can't tell from that code snippet. I see nothing inherently wrong with that code other than you are not doing anything with 'messageTemp'. If the ESP32 is restarting it could memory corruption. What does the console show?

[Danois90]

Usually this is related to the limited stack available to callback's/functions in general. The conversion makes no sense, though.. You could simply cast "message" to "char*" and use it without creating a copy of it. But impossible to give good advice based on a snippet.

[J-M-L]

The compiler likely gets rid of this anyway as messageTemp is a local variable unused after.

@filipekerplunk

char messageTemp[length];
  for (int i = 0; i < length; i++) {
    //dbSerialPrint((char)message[i]);
    messageTemp[i] = (char)message[i];
  }

If this is not the full code, You also might be short in stack space if length is big.
You also might want to allocate an extra char for a trailing null if you want to deal with messageTemp as a cString later on

[alto777]

@filipekerplunk

  dbSerialPrint("Mensagem chegou no topico: ");
  dbSerialPrint(topic);
  dbSerialPrint(". Tamanho: ");
  dbSerialPrint(length);

Message arrived on topic.

Size.

Portuguese!

a7

[filipekerplunk]

It's probably due to the stack because it stops after I declare the char[]. The message is large, about 50000 bytes. Tested with a debug.

void callback(char* topic,  unsigned char* message, unsigned int length) {
  dbSerialPrint("Mensagem chegou no topico: ");
  dbSerialPrint(topic);
  dbSerialPrint(". Tamanho: ");
  dbSerialPrint(length);
  
  char messageTemp[length];
  dbSerialPrint("Converting: "); //this line dont show 
  for (int i = 0; i < length; i++) {
    //dbSerialPrint((char)message[i]);
    messageTemp[i] = (char)message[i];
  }
}

I can't work only with unsigned char* because then I need to save to a SPIFFS file and by SPIFFS it doesn't accept print(unsigned char* variable)

[alto777]

Show us where you need the char *.

You can use a cast. Probably.

At the bit level there’s no difference, so we must tell a lie…

a7

[PieterP]

@filipekerplunk

I can't work only with unsigned char* because then I need to save to a SPIFFS file and by SPIFFS it doesn't accept print(unsigned char* variable)

File::print might not accept unsigned char *, but File::write does: arduino-esp32/libraries/FS/src/FS.h at 21947ebe76cf6380fe06aa885390b86cc477709a · espressif/arduino-esp32 · GitHub

In this specific case, it would be valid to use a cast:

void callback(char* topic,  unsigned char* message, unsigned int length) {
  ...
  file.print(reinterpret_cast<const char *>(message));
}

Whether it's a good idea to just assume that the message is properly null-terminated etc. is a different issue.

Warning: reinterpret_cast makes it very easy to shoot yourself in the foot, there are very strict rules about when it's allowed, see reinterpret_cast conversion - cppreference.com. In this specific case, you're casting to char which is an exception to the rules, you cannot do this for general types.

[J-M-L]

@filipekerplunk

The message is large, about 50000 bytes.

That’s a large request for the stack. As a quick workaround you could make a large global variable buffer that you keep around for those manipulation (if they are really needed).

[filipekerplunk]

I'm facing a big dilemma/problem

This callback gives me a json via mqtt, which I need to write to SPIFFS and deserialize with ArduinoJson.

It actually works like this:

class Locker
{
  public:
    String id;
    String status;
    String dataStatus;
    int number;
    String document;
    String keycode;
    String bookingNumber;
    String nrOfDays;
};

Locker lockers[99];

void callback(char* topic,  unsigned char* message, unsigned int length) {
  dbSerialPrint("Mensagem chegou no topico: ");
  dbSerialPrint(topic);
  dbSerialPrint(". Tamanho: ");
  dbSerialPrint(length);

  String messageTemp;
  //json = "";
  for (int i = 0; i < length; i++) {
    //dbSerialPrint((char)message[i]);
    messageTemp += (char)message[i];
  }
  criarClasse(messageTemp); //desserialize
  gravarArquivo(messageTemp); //write in spiffs
  messageTemp = "";
}


void criarClasse(String json)
{

  StaticJsonDocument<256> filter;

  JsonObject filter__ = filter.createNestedObject("*");
  filter__["document"] = true;
  filter__["keycode"] = true;
  filter__["number"] = true;
  filter__["status"] = true;
  filter__["nrOfDays"] = true;
  filter__["bookingNumber"] = true;
  filter__["dataStatus"] = true;

  DynamicJsonDocument doc(10000);

  DeserializationError error = deserializeJson(doc, json, DeserializationOption::Filter(filter));

  if (error) {
    dbSerialPrint(F("deserializeJson() failed: "));
    dbSerialPrintln(error.f_str());
    //return;
  }

  JsonObject root = doc.as<JsonObject>();
 
  int count = -1;
  for (JsonPair kv : root)
  {
    count++;
    lockers[count].id = kv.key().c_str();
    lockers[count].document = (const char*)doc[lockers[count].id]["document"];
    lockers[count].keycode = (const char*)doc[lockers[count].id]["keycode"];
    lockers[count].number = doc[lockers[count].id]["number"];
    lockers[count].status = (const char*)doc[lockers[count].id]["status"];
    lockers[count].nrOfDays = (const char*)doc[lockers[count].id]["nrOfDays"];
    lockers[count].bookingNumber = (const char*)doc[lockers[count].id]["bookingNumber"];
    lockers[count].dataStatus = (const char*)doc[lockers[count].id]["dataStatus"];
  }

void gravarArquivo(String gravar)
{
  //Write SPIFFS
  File f = SPIFFS.open("/myspiffs.json", "w");
  f.print(gravar);
  f.close();
}

Even filtering out the data that comes with just the fields I want, I sometimes seem to have stack overlay issues where deserialization fails. So I was trying to replace the Strings with something else, char* or even const char*. The unsigned char* itself could do it too.

Changing from String to const char* would resolve this stack fragmentation a bit?

Is there any difference between String and const char*?

Thanks!

[J-M-L]

@filipekerplunk

Is there any difference between String and const char*?

Yes… a const char * is just a pointer to an area in memory where you keep a character constant. So it has no storage. A String, or more precisely an instance of the String class, has a infinite (memory limited) storage allocated that grows as you add stuff into the String. That storage is not allocated on the stack though, but in the heap.

The question is more what do you need to keep to describe a locker. You don’t want to keep pointers into a transient char buffer, you need to duplicate the json data.

[filipekerplunk]

But if I changed String messageTemp to const char*, and changed the function parameters, would it alleviate the heap?

[J-M-L]

As I said, there is no data storage associated with the pointer. The buffer you get in the callback as parameter is not guaranteed to stay valid forever and the json document is transient so if you were to keep pointers into that buffer they would become stale likely as soon as you exit the function.

You need to duplicate what is in the json data structure into memory you own and control

[drmpf]

There are alternative Json libraries like jRead and it companion jWrite which work on a fixed buffer.
The ArduinoJson allocates memory and does not always free it when you would expect.
Global allocation seems to work best for ArduinoJson