Store and compare strings that can have different lengths

Hi Community,

I have a callback function, that receives messages from an MQTT Server where the ESP8266-Module, the code is run on, has subscribed to.
The code:

void callback(char *topic, byte *payload, unsigned int length) {
  //parse message
  char outputBuffer[length];  //String zur Speicherung der Message
  int i = 0;
  for (i; i < length; i++) {
    outputBuffer[i] = (char)payload[i];
  }
  outputBuffer[i] = '\0';  //terminate string
  Serial.print("Full message: ");
  Serial.println(outputBuffer);
  //do something reasonable with the message like controlling something
  }

My issue is: Upon restart, the MQTT Server issues the last message again to the subscribers, which can cause "duplicates" on the ESP8266 side. To avoid this, I would like to store the string received and compare upon receiving a new message, if the messages were identical.

I assume I will end with something like:

if !(strcmp (outputBuffer,storedmessage) != 0)

but my problem is, that the lengths can be different each time.
How can I store the message (or calculate it's hash key and compare them) in such a case?

Have a look at strstr() it might make your life easier.

Why do you think that the messages having a different length is a problem ? Declare storedmessage as a global that can hold the maximum size of the MQTT message

Your outputBuffer array is only declared large enough to hold the number of bytes received so when you add the string terminator you are writing outside of the array

1 Like

Is the payload a string? The callback parameters seem to say it's a pointer to a series of bytes with a given length.

Could that series of bytes include zero values? If so, treating it as a string won't work, because strcmp() will stop comparing when it reaches a zero byte. This could make your code think that the messages are the same when in fact they are not.

I tried to consider the given tips and came up with:

  1. Restricting the length of the message on the MQTT Server to 32 Chars
  2. increasing local buffer to hold the termination char
char lastcommand[32];          //global buffer to store last msg


void callback(char *topic, byte *payload, unsigned int length) {

  //parse message
  char outputBuffer[length+1];
  int i = 0;
  for (i; i < length; i++) {
    outputBuffer[i] = (char)payload[i];
  }

  //verify if string is identical with last one received
  int match = strcmp (outputBuffer, lastcommand);
  if (match == 0) return;      //quit function if strings match
  memset(lastcommand, 0, sizeof lastcommand);  //clear old copy
  strcpy(lastcommand, outputBuffer); //copy current message to buffer 

  outputBuffer[i] = '\0';  //terminate received string
 Serial.print("Full message: ");
 Serial.println(outputBuffer);
  //do something reasonable with the message like controlling something

However: I now changed the problem, a little by introducing a max-length which only considers the first 32 chars :wink:

You seem to be making things difficult for yourself
Here is how I see it

  • Declare outputBuffer large enough to hold the largest possible MQTT message. Ignore the length of the received message
  • Copy the received message to the buffer. I would use memset(), then terminate the string
  • Compare the previous string to the received string using strcmp()
  • if they are different then do what you need
  • If they are the same then do what you need
  • Either way, copy the new string to the old string using strcpy()

I am a little confused,

  • Copy the received message to the buffer. I would use memset(), then terminate the string

How do I use memset() to copy the buffer ?

I think I do almost everything you reccomend except for declaring the buffer with the Max-Size of the MQTT Message and I am looping through the array and copy each char but then I compare the strings using strcmp() and strcopy() to save it, if it was a new one.

If I get you right I could remove the loop by copying the full array and remove the memset to 0 as it seems to be not necessary, or am I missing something ?

An example from one of my MQTT test sketches

void callback(char* topic, byte* payload, unsigned int length)
{
  memcpy(messageBuffer, payload, length);
  messageBuffer[length] = '\0';
  Serial.println(messageBuffer);
}

Note that messageBuffer here is a global variable declared like this

char messageBuffer[50];

It could just was well `be local variable as long as its scope suits your purpose

@UKHeliBob got it, thanks for the support. You wrote "I'd use memset [to copy the array]" while you actually used memcpy but now it is clear.

The final code now looks like this:

char lastcommand[128];          //global buffer to store last msg

//callback method, dealing with incoming mqtt message
void callback(char *topic, byte *payload, unsigned int length) {

  //block processing of message for ~30sec after reboot of ESP ... 

  //parse message
  char outputBuffer[length];  //String zur Speicherung der Message
  memcpy(outputBuffer, payload, length);
  outputBuffer[length] = '\0';  //terminate received string
  
  //verify if string is identical with last one received
  int match = strcmp (outputBuffer, lastcommand);
  if (match == 0)    return;      //quit function if strings match and do not process it
  strcpy(lastcommand, outputBuffer); //copy current message to buffer 

  //do something with the message
}

Setting the "lastcommand[]" to 0 using memset, seems not to have any impacts.
The String ABCD and ABC are considered as two different messages, even without clearing the array.

My apologies for the mistake but I am glad that you saw through it. Using memcpy() seems much neater than a for loop and you can then use the value of the length parameter to add the trailing '\0'

A minor point, but it could be important at some time,, they are not Strings (objects of the String library) but strings (zero terminated arrays of char)

As to ABCD and ABC being different, it is, of course, what I would expect and is a result of terminating the array correctly and not writing outside if its bounds

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.