Extracting variable value from MQTT received message

I have a variable, called shutdown, whose value is being written to my microcontroller via a UART connection to a modem. I would like to take the received message, extract the integer value given for shutdown then make my global variable called "shutdown" equal to it.

The message format is rather complicated, how might one go about extracting this? On the serial monitor the message coming in looks like:

15:34:09.044 -> +QMTRECV: 1,0,"motorbeta/shutdown","shutdown=1"

shutdown=atoi(&msg[strlen(msg)-2]);
1 Like

Knowing the topic is always shutdown a 1 or 0 as payload should suffice.

You can find the index of '=' and the index of the following ' " ', this will give you a location and length.

@Idahowalker you certainly are correct, I don't have any control over the format of the incoming message though.

I should mention that to get the shutdown message, I have to send a AT command then I wait for the end of the message and exit this while loop. The code I am using to put the message on the serial monitor looks like the first block. I am trying to modify it to include your suggestion but it seems like my data types aren't meshing well as I am getting:

exit status 1
cannot convert 'String' to 'const char*' for argument '1' to 'size_t strlen(const char*)'

Original Code

void atsendbasic(String command, int fallout3, int fallout4){
   unsigned long initialtime4 = millis();
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((!Serial2.available() && millis() - initialtime4 <= fallout3) || millis() - initialtime4 <= fallout4) continue;
    while ((Serial2.available()&& millis() - initialtime4 <= fallout3) || millis() - initialtime4 <= fallout4) {
      char feedback = Serial2.read();
      Serial.print(feedback);
    }
}

Modified Code

void atreceivebasic(String command, int fallout3, int fallout4){
   char feedback;
   unsigned long initialtime5 = millis();
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((!Serial2.available() && millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) continue;
    while ((Serial2.available()&& millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) {
      feedback = Serial2.read();
      Serial.print(feedback);
    }
    String feedbackstr = String(feedback);
    int shutdownr=atoi(&feedbackstr[strlen(feedbackstr)-2]);
}

assuming you may want to extract other variables

1
3
not found

const char t [] = "15:34:09.044 -> +QMTRECV: 1,0,\"motorbeta/shutdown\",\"shutdown=1\"";
const char u [] = "15:34:09.044 -> +QMTRECV: 1,0,\"motorbeta/shutdown\",\"shutdown=3\"";
const char v [] = "15:34:09.044 -> +QMTRECV: 1,0,\"motorbeta/shutdown\",\"motobeta=3\"";

// -----------------------------------------------------------------------------
int
getValue (
    char *s,
    const char *lbl,
    int  *val )
{
    char fmt [40];
    sprintf (fmt, "%s=", lbl);

    char *q = strstr (s, fmt);
    if (NULL == q)
        return 0;

    sprintf (fmt, "%s=%%d", lbl);
    sscanf (q, fmt, val);
    return 1;
}

// -----------------------------------------------------------------------------
void
setup (void)
{
    Serial.begin (9600);

    int shutdown;

    if (getValue ((char*)t, "shutdown", &shutdown))
        Serial.println (shutdown);
    else
        Serial.println ("not found");

    if (getValue ((char*)u, "shutdown", &shutdown))
        Serial.println (shutdown);
    else
        Serial.println ("not found");

    if (getValue ((char*)v, "shutdown", &shutdown))
        Serial.println (shutdown);
    else
        Serial.println ("not found");
}

void
loop (void)
{
}
1 Like

I am somewhat of a beginner so I am a little confused by this. I am confused as to what this block of code does?

if you are not sure of the overall message format but know the critical text is of the format "shutdown=1" you could use the string tokensizer to

  1. split the string using delimiters " and =
  2. when you find the shutdown token you know the next token is the integer required
  3. get next token convert the string to an integer and exit the tokenizer loop

e.g. sample code

#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="15:34:09.044 -> +QMTRECV: 1,0,\"motorbeta/shutdown\",\"shutdown=1\"";
  printf ("Splitting string \"%s\" into tokens:\n",str);
  char *pch = strtok (str,"\"=");
  int shutdown_found=0;
  while (pch != NULL)
  {
    printf ("string found %s\n",pch);
    // if shutdown was last token read integer
    if(shutdown_found) {
        int x=atoi(pch);
        printf("shutdown found = %d\n", x);
        return 1;
    }
    // look for shutdown token
    if(strcmp(pch,"shutdown")==0)shutdown_found=1;
     pch = strtok (NULL, "\"=");
  }
  return 0;
}

a run gives

Splitting string "15:34:09.044 -> +QMTRECV: 1,0,"motorbeta/shutdown","shutdown=1"" into tokens:
string found 15:34:09.044 -> +QMTRECV: 1,0,
string found motorbeta/shutdown
string found ,
string found shutdown
string found 1
shutdown found = 1
1 Like

it's a sub-function that can be given a string, a label (e.g. "shutdown") and a pointer to an integer to set to the value specified in the string by the label (e.g. "shudown=1"). the sub-function returns a 1 if it finds the label in the string along with setting the val, otherwise it returns 0 meaning that the string didn't have the label.

it uses sprintf to format strings used to process the received message string

1 Like

@gcjr I do see this objective and this seems to align pretty well with what I need. But man I am having trouble understanding. I guess this is because my cpp is pretty weak... :frowning:

sprintf (fmt, "%s=", lbl);

I have been looking at the example below regarding this function, and I can follow it but don't know how to apply to our example. Especially this part "%s=" I could understand if it was simply s which is meant to be the entire message. But this is altering it in some way I don't understand.

// Example program to demonstrate sprintf()
#include <stdio.h>
int main()
{
	char buffer[50];
	int a = 10, b = 20, c;
	c = a + b;
	sprintf(buffer, "Sum of %d and %d is %d", a, b, c);

	// The string "sum of 10 and 20 is 30" is stored
	// into buffer instead of printing on stdout
	printf("%s", buffer);

	return 0;
}

I have also tried this but it seems there is a problem with it when I attempt to compile it using a serial.read for str[]

Like so:

void atreceivebasic(String command, int fallout3, int fallout4){
   char str[];
   unsigned long initialtime5 = millis();
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((!Serial2.available() && millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) continue;
    while ((Serial2.available()&& millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) {
      str = Serial2.read();
      Serial.print(str);
    }
      printf ("Splitting string \"%s\" into tokens:\n",str);
      char *pch = strtok (str,"\"=");
      int shutdown_found=0;
      while (pch != NULL){
      printf ("string found %s\n",pch);
      // if shutdown was last token read integer
      if(shutdown_found) {
          int x=atoi(pch);
          shutdown = x;
          printf("shutdown found = %d\n", x);
          return 1;
      }
      // look for shutdown token
      if(strcmp(pch,"shutdown")==0)shutdown_found=1;
      pch = strtok (NULL, "\"=");
      }
      return 0;
    
}

I get the error:

exit status 1
storage size of 'str' isn't known

So if I make str[100] then I get the error:

exit status 1
incompatible types in assignment of 'int' to 'char [100]'

if you know the string contains "shutdown=1"` the simplest solution is from post #2

// strtok - search sting for shutdown=x - get value of x
#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="15:34:09.044 -> +QMTRECV: 1,0,\"motorbeta/shutdown\",\"shutdown=1\"";
  printf("shutdown  = %d\n",atoi(&str[strlen(str)-2]));
}

a run gives

shutdown  = 1
1 Like

sprintf() formats strings similar to printf(). it's being used to create patterns to determine if the desired value is present in the received string. the code can't simply look for the label, "shutdown" since it appears elsewhere, "motorbeta/shutdown", an "=" must be appended.

sprintf() is then used a 2nd time in sscanf() to more directly extract the value.

I am able to run and understand this example code, as it is pretty simple.

As soon as I try to apply it to a serial.read though I start to have problems. Because unlike the example code I need to make the str[] value isn't known until we read the serial data:

void atreceivebasic(String command){
   
   char str[];
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((Serial2.available()) {
      str = Serial2.read();
    }
    shutdown = atoi(&str[strlen(str)-2]);
    
}

this gives me the error:

exit status 1
storage size of 'str' isn't known

So I changed to:

void atreceivebasic(String command){

   char* feedback;
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((Serial2.available()) {
      char feedback = Serial2.read();
      Serial.print(feedback);
    } 
      char* str = feedback;
      shutdown = atoi(&str[strlen(str)-2]); 
      Serial.println(shutdown);
}

but on my microcontroller I am seeing this error when I run and attempt to read my serial monitor sending the data:

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

then the microcontroller reboots

I should note that the Serial.print(feedback) outputted this just before the failure. The timestamps are not part of the outputs just new lines on the serial monitor:

15:45:17.907 -> AT+QMTPUBEX=1,0,0,0,"motorbeta/heartbeat","{"pulse":1,"secondary":2}"
15:45:17.907 -> OK
15:45:17.907 -> 
15:45:17.907 -> +QMTPUB: 1,0,0
15:45:17.907 -> 
15:45:17.907 -> +QMTRECV: 1,0,"motorbeta/shutdown","shutdown=1"

So 1 message with 5 lines, only the last of which I care about. The rest are just command echoes and success confirmations.

your first code of post #15 had a number of errors as indicated below which now compiles

void atreceivebasic(String command){
   
   char str[100];    //  you have to specifiy the size of array str
   int arrayIndex=0; // <<<  needed to access array
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while (Serial2.available()) {   // <<  too many (
      str[arrayIndex++] = Serial2.read();  // << you need ato access an array element
    }
    int shutdown = atoi(&str[strlen(str)-2]);  // << specify tyep of shutdown   
}

in your second code you define feedback as a char* (pointer to char) but don't point it at anything
as it is a local variable it could contain any value and when use would corrupt memory
you then assign the value of feedback (which is anything) to str (a char *) and then use str in the following statments which points somewhere in memory and then crashes the program

it C programs crash (like yours) look for pointer problems, array overflow, etc etc

1 Like

This does compile and run. And when a message is received it doesn't overflow like you mentioned so a step in the right direction. It even outputs a 0, although it should be outputting a 1.

Below is copy pasted from the serial monitor output. The custom function is being executed between the lines "sending awake status" and "hold up" as you can see there is a 0 there, but interestingly the entire rest of the message is printed when I read the serial monitor later in the program. It seems to me that the function is only analyzing one or a few characters of the message, unlike my previous approach.

16:49:27.922 -> sending awake status
16:49:27.922 -> 0
16:49:28.446 -> hold up
16:49:28.446 -> A
16:49:33.460 -> T+QMTPUBEX=1,0,0,0,"motorbeta/heartbeat","{"pulse":1,"secondary":2}"
16:49:33.460 -> OK
16:49:33.460 -> 
16:49:33.460 -> +QMTPUB: 1,0,0

Below is the exact place in the code where I am calling our custom function "atreceivebasic"

 Serial.println("sending awake status");
 //send awake status
 atreceivebasic("AT+QMTPUBEX=1,0,0,0,\"motorbeta/heartbeat\",\"{\"pulse\":1,\"secondary\":2}\"\r\n",60000,5000);
 delay(500);
 Serial.println("hold up");
 char subscription = Serial2.read();
 Serial.println(subscription);

with my previous code which could only read serial monitor and not assign shutdown to an integer value I was seeing:

15:45:17.907 -> AT+QMTPUBEX=1,0,0,0,"motorbeta/heartbeat","{"pulse":1,"secondary":2}"
15:45:17.907 -> OK
15:45:17.907 -> 
15:45:17.907 -> +QMTPUB: 1,0,0
15:45:17.907 -> 
15:45:17.907 -> +QMTRECV: 1,0,"motorbeta/shutdown","shutdown=1"

For reference this was the function I was using to read my serial monitor

void atsendbasic(String command, int fallout3, int fallout4){
   unsigned long initialtime4 = millis();
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
    while ((!Serial2.available() && millis() - initialtime4 <= fallout3) || millis() - initialtime4 <= fallout4) continue;
    while ((Serial2.available()&& millis() - initialtime4 <= fallout3) || millis() - initialtime4 <= fallout4) {
      char feedback = Serial2.read();
      Serial.print(feedback);
    }
}

the example of post #16 determines the value of shutdown as 1 but does not do anything with it in the example shutdown is a local variable, should it be global or returned as the function result?

It does output a 1 in the test script you posted but when I use it to read my actual application's serial data I see a 0, and the other artifacts I posted about post #17. I guess what I am getting at is input this entire 5line sequence to the function and it will fail. I know this is an expansion of the scope of the code, but this is what is leading to my failure...

int shutdown should be a global variable if I understood you correctly, and I have defined it as such prior to my setup loop().

Is there a way to adjust this code to the serial data I am seeing, and accomplish the task at hand? Extracting the value of shutdown and assigning it to a global variable.

15:45:17.907 -> AT+QMTPUBEX=1,0,0,0,"motorbeta/heartbeat","{"pulse":1,"secondary":2}"
15:45:17.907 -> OK
15:45:17.907 -> 
15:45:17.907 -> +QMTPUB: 1,0,0
15:45:17.907 -> 
15:45:17.907 -> +QMTRECV: 1,0,"motorbeta/shutdown","shutdown=1"

Before I explore @Idahowalker code wanted to show that with the code below I am getting no errors just not able to detect the shutdown=1 and reporting a 0. Again it is likely failing because I am analyzing this 5 line message instead of the single line message. Serial output is looking like this:

21:05:23.510 -> sending awake status
21:05:28.524 -> 0
21:05:29.040 -> hold up
21:05:29.040 -> ⸮

for this section of code:

 Serial.println("sending awake status");
 //send awake status
 atreceivebasic("AT+QMTPUBEX=1,0,0,0,\"motorbeta/heartbeat\",\"{\"pulse\":1,\"secondary\":2}\"\r\n",60000,5000);
 delay(500);
 Serial.println("hold up");

the atreceivebasic function looks like this:

void atreceivebasic(String command, int fallout3, int fallout4){
   unsigned long initialtime5 = millis();
   char feedback;
   Serial2.print(command);
   Serial2.flush();  // Wait until it is all sent
   while ((!Serial2.available() && millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) continue;
   while ((Serial2.available()&& millis() - initialtime5 <= fallout3) || millis() - initialtime5 <= fallout4) {
      feedback = Serial2.read();
    }   
      char *pfeedback = &feedback;
      int shutdown = atoi(&pfeedback[strlen(pfeedback)-2]);
      Serial.println(shutdown);
}