Sscanf bug, what I am doing wrong

I am trying to use sscanf and had some succes, but when the program grew and I finally thought to get it... I am getting strange behaviour. Now it prints 5 zeroes but I have also seen that every byte print would print the entire message + a zero.

char *message = "" ;

uint16_t SLAVE_ID ;
uint16_t COMMAND ;
uint16_t IO_ID ;
uint16_t DATA1 ;
uint16_t DATA2 ;

bool getMessage()
{
    static int index = 0 ;

    if( Serial.available() > 0 )
    {    
        char c = Serial.read() ;

        if( c == '\n')
        {
            //message[index++] = c ;
            message[index] = 0 ;
            index = 0 ;
            return true ;
        }
        else if( c >= ' ' )   // discard all non printable characters except newline
        {
            message[index++] = c ;
        }
    }
    return false; 
}


void setup()
{
    Serial.begin( 9600 ) ;
}

void loop()
{
    if( getMessage() )
    {
        sscanf( message, "%d,%d,%d,%d,%d", 
            &SLAVE_ID, 
            &COMMAND, 
            &IO_ID, 
            &DATA1,
            &DATA2 ) ; // message = 1,2,3,4

        Serial.println(F("message received")) ;
        //delay(1000);
        Serial.println( SLAVE_ID ) ;
        Serial.println( COMMAND ) ;
        Serial.println( IO_ID ) ;
        Serial.println( DATA1 ) ;
        Serial.println( DATA2 ) ;

        //processMessage() ;
    }

    //updateServos() ;
    //readInputs() ;
}

Unless there is a compiler bug in play, I must be doing something wrong. I use uint16_t variables, dit not forget the & and the amount of %d matches the amount of variables.

What is it that I am doing wrong?

Kind regards,

Bas

Change your buffer to

 char message[32];

The way you did it reserved no room for the characters you gather.

BTW

   Unless there is a compiler bug in play,

last thing you should suspect. It's 2022, the compiler is ever less likely to be the problem. :wink:

a7

And replace %d with %hu

1 Like

Huge problem. You will be corrupting memory.

consider

bool getMessage()
{
    if( Serial.available() > 0 ) {
        int n = Serial.readBytesUntil ('\n', message, sizeof(message)-1);
        message [n] = '\0';
        return true ;
    }
    return false;
}

Do you have your "Compiler warnings" (in Preferences...) turned down below 'All'? You should have gotten a warning on your first line. It's not very clear but is warning of the real problem: using a read-only empty string as your buffer address.

sketch_oct09a.ino:1:17: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
 char *message = "" ;
                 ^~
1 Like

readBytesUntil() blocks.

a7

yes it will block until the '\n', but when is a string not sent as a consecutive sequence of chars terminated by a \r or \n?

if you enter a string in the serial monitor, the string isn't sent until you hit the enter key

Yeah, no

Thnx, implemented and fully operational here!

No I haven't. I used to be too lazy for that. I usually stay away from complicated pointer logic and in practice I don't this type of "strange behaviour", but I will turn them on. If it doesn't hurt... must be good for ya.

The blocking isn't so much an issue, I think... I wont be trying it out though. This code is for an arduino slave which needs to set relays, move servos and debounce and relay inputs to a master.

A complete transmission is propably less than 20 bytes or so, depending on the values. There is the catch that I am working on 9600 BPS.

I am trying to remember why I wanted to send comma separated data in ASCII format instead of just 5 whole bytes but I can't remember. Atleast I get to debug using the serial monitor.

Kind regards,

Bas

I put your code (with the buffer fix) into the wokwi simulator to check a few things about which I now know more.

Even if the entire line is available, it still comes in at the character rate that 9600 baud supports

Play with blocking and non-blocking versions in the simulator at this link:

block test simulation

Change the one BLOCKS defined at the top to select non-blocking or blocking code.

// https://forum.arduino.cc/t/sscanf-bug-what-i-am-doing-wrong/1046254

# define BLOCKS false      // select blocking (true) or non-blocking (false) code 

char message[32];

uint16_t SLAVE_ID ;
uint16_t COMMAND ;
uint16_t IO_ID ;
uint16_t DATA1 ;
uint16_t DATA2 ;

bool gthering = false;
unsigned long aTime;

unsigned loopCounter;

# if BLOCKS

bool getMessage()
{
    if( Serial.available() > 0 ) {
        aTime = millis();
        int n = Serial.readBytesUntil ('\n', message, sizeof(message)-1);
        message [n] = '\0';

        aTime = millis() - aTime;
        return true ;
    }
    return false;
}

# else // non-blocking version

bool getMessage()
{
    static int index = 0 ;

    if( Serial.available() > 0 )
    {  

// turn on counter when we begin gathering
      if (!loopCounter) loopCounter = 1;

        char c = Serial.read() ;

        if( c == '\n')
        {
            //message[index++] = c ;
            message[index] = 0 ;
            index = 0 ;
            return true ;
        }
        else if( c >= ' ' )   // discard all non printable characters except newline
        {
            message[index++] = c ;
        }
    }
    return false; 
}

# endif

void setup()
{
    Serial.begin( 9600 ) ;
    Serial.print("wake up!  ");
# if BLOCKS
    Serial.println("blocking code version");
# else
    Serial.println("non-blocking code version");
#  endif
}

void loop()
{
    if (loopCounter) loopCounter++;    // we in the middle of gathering a complete line

    if( getMessage() )
    {
        sscanf( message, "%u,%u,%u,%u,%u", 
            &SLAVE_ID, 
            &COMMAND, 
            &IO_ID, 
            &DATA1,
            &DATA2 ) ; // message = 1,2,3,4

        Serial.print("message received ");

# if BLOCKS
      Serial.print(" and getting it blocked for ");
      Serial.print(aTime);
      Serial.println(" milliseconds ");

# else // doesn't block

        Serial.print("and the loop got attention while it was coming in ");
        Serial.print(loopCounter);
        Serial.println(" times.");
# endif
        loopCounter = 0;

        //delay(1000);

        Serial.println( SLAVE_ID ) ;
        Serial.println( COMMAND ) ;
        Serial.println( IO_ID ) ;
        Serial.println( DATA1 ) ;
        Serial.println( DATA2 ) ;

        //processMessage() ;
    }

    //updateServos() ;
    //readInputs() ;
}

In the real world, you get to decide. In the real world, not all serial data gets held until a full line is ready to transmit, and even if it were true, there's still a block when the readBytesUntil() is called. Of X characters times Y ms, 25 ms for a package of 5 4-digit values at 9600 baud

In the worst case, the sender drops the ball mid-message and the receiver is left hanging for as long as the serial time out constant - 1000 ms by default, adjustable with the setTimeout() function the Serial object provides.

In some programs, 25 ms is nothing. Neither might be anything a full 1000 ms block. In others, 25 ms is a virtual eternity, to go away for which time is simply not acceptable.

Depends, like so many decisions. On whatever.

You went to some trouble to code a non-blocking getMessage() and it also let you do some filtering, another feature that might not be important were you only talking about the serial monitor.

The wokwi simulator, can't pass up any opportunity to promote its use. Free, and if you prefer, anonymous:

a7

Thank you for your elaborate answer

25ms is actually just a tad too much. I need to update the servo's about every 20ms. So I'll stick to the non blocking code.

I tried out the code on wokwi, but blocking or non blocking didn't seem to make a difference. Than I tried both on an arduino and got the same results.. Than I noticed a tiny mistake

#if BLOCKS should be #if defined/#ifdef BLOCKS

But Now i can clearly see the difference between the two :wink:

Kind regards,

Bas

if it's 25 msec at 9600 bps, it's just 2.1msec at 115200 bps

1 Like

I'm not sure why or how it didn't work for you. Naturally I do not want to have a mistaken demo.

At this latitude and longitude:

Execute unmodifed out of the box: (click on "block test simulation" link, where it is preset to non-blobking:

  # define BLOCKS false      // select blocking (true) or non-blocking (false) code 

output with 100,200,300,400,500 as the input line:

wake up! non-blocking code version
message received and the loop got attention while it was coming in 6776 times.
100
200
300
400
500



Then changing just one line at the top

 # define BLOCKS true      // select blocking (true) or non-blocking (false) code 

output with 100,200,300,400,500 as the input line:

wake up! blocking code version
message received and getting it blocked for 20 milliseconds
100
200
300
400
500


I carefully did exactly the same with a real UNO just now.

I repeated my test on another machine that was not logged in as a wokwi use. Same same.

In any case, I am glad you got the two to show themsleves no matter how you figured it out.

Of course raising the baud rate reduces theduration of the blockage, but still depends on the message, or whatever you call it, to come in as one complete line. Since this is not always the case, you leave open the possibility of a failure at the transmitter making the blockage very worse.

Why block? For 25, 2.5 or 0.25 ms? When you don't need to.

a7

Why block? For 25, 2.5 or 0.25 ms? When you don't need to.

I did say

So I'll stick to the non blocking code.

So il be fine.

That baudrate is chosen on 9600 because I need to use software serial libary. I was not entirely sure if software serial on 115200 would give me any instabillity issues. As I intend to send only 1 message per second, I thought that 9600 would simply suffice.

About blocking code, I realized that I made another mistake in the transmitting part. I haven't told yet, but I am also implementing Rs485. So in order to control the direction line, I thought I might as well use Serial.flush(); but that also blocks the code..

    digitalWrite( rs485dir, HIGH ) ;

    Serial.print(0) ;           Serial.write(',') ;                 //  ID master
    Serial.print(sendInput) ;   Serial.write(',') ;                 //  input instruction
    Serial.print(i) ;           Serial.write(',') ;                 //  ID of IO
    Serial.println( state ) ;                                       //  state of IO
    Serial.flush() ;                                               // this one got to go

    digitalWrite( rs485dir, LOW ) ;                                // this gotta go too

If I understand correctly I must use the method availableForWrite()
Can I simply put this in loop()? I am aiming to release the line as soon as the last byte is sent without blocking.

void loop()
{
    task() ;
    Othertask() ;
    etc() ;

    if( Serial.availableForWrite() == 0 ) 
    {
        digitalWrite( rs485dir, LOW ) ; // release transmitt line after last bytes are sent
    }
}

Regards,

Bas

EDIT: Wait, I am going to test this first.

I misinterpreted this function. It works the other way around. If there is nothing to send anymore and the transmitt buffer is empty, this method will return 63. Which is the size of the Tx buffer - 1.

This code example

uint32_t beginTime ;
uint32_t endTime ;
const int rs485dir = 13 ;

void setup()
{
  pinMode(rs485dir, OUTPUT) ;
  Serial.begin(9600) ;
  sendBlockingMessage() ;
  delay(500);
  sendNonBlockingMessage() ;
}


void sendBlockingMessage()
{
   beginTime = millis() ;
   for( int i = 0 ; i < 25 ; i ++ )
   {
       Serial.write('X') ;
   }
   Serial.println() ;
   Serial.flush() ;
   endTime = millis() ;

   Serial.print("blocking time: ");
   Serial.println( endTime - beginTime ) ; 
}

void sendNonBlockingMessage()
{
   beginTime = millis() ;
   for( int i = 0 ; i < 25 ; i ++ )
   {
       Serial.write('X') ;// send 25x
   }
   Serial.println() ;
   
   digitalWrite(rs485dir, HIGH ) ;
}

void loop()
{   
  if( Serial.availableForWrite() == 63 
  && digitalRead( rs485dir ) )
  {
    endTime = millis() ;
    Serial.print("non blocking time: ");
    Serial.println( endTime - beginTime ) ;
    digitalWrite( rs485dir, LOW) ;
  }
}

gives this output

XXXXXXXXXXXXXXXXXXXXXXXXX
blocking time: 27
XXXXXXXXXXXXXXXXXXXXXXXXX
non blocking time: 26

Kind regards,

Bas

You are not measuring block or no block, you are measuring how long it takes the buffer to empty.

It's always going to take X characters times Y ms / character for the buffer to empty.

Try this what shows the block/no block difference:

direct measurement of blackage

simpler sketch

// https://forum.arduino.cc/t/sscanf-bug-what-i-am-doing-wrong/1046254

const int rs485dir = 13 ;


uint32_t myBeginTime ;
uint32_t myEndTime ;
uint32_t anotherBeginTime ;
uint32_t anotherEndTime ;
uint32_t bufferEmptyTime ;

void setup()
{
  Serial.begin(9600) ;
  Serial.println("block no block\n");

  pinMode(rs485dir, OUTPUT) ;

myBeginTime = millis();

  sendBlockingMessage() ;

myEndTime = millis();

   Serial.print( "                          my blocking time measurment : ");
   Serial.println(myEndTime - myBeginTime ) ; 

  delay(500);

myBeginTime = millis();

  sendNonBlockingMessage() ;

myEndTime = millis();
}

void sendBlockingMessage()
{
   for( int i = 0 ; i < 25 ; i ++ )
   {
       Serial.write('X') ;
   }
   Serial.println() ;

anotherBeginTime = millis();

   Serial.flush() ;

anotherEndTime = millis();
}

void sendNonBlockingMessage()
{
   for( int i = 0 ; i < 25 ; i ++ )
   {
       Serial.write('X') ;// send 25x
   }
   Serial.println() ;

  digitalWrite(rs485dir, HIGH ) ;
}

void loop()
{  
  if( Serial.availableForWrite() == 63 
  && digitalRead( rs485dir ) )
  {
    bufferEmptyTime = millis() ;

    Serial.print( "                          my non blocking time measurment : ");
    Serial.println(myEndTime - myBeginTime ) ; 


    Serial.print("buffer is empty after: ");
    Serial.println(bufferEmptyTime - myBeginTime ) ;

    Serial.print("serial flush took");
    Serial.println(anotherEndTime - anotherBeginTime ) ;




    digitalWrite( rs485dir, LOW) ;


  }
}

which reports

XXXXXXXXXXXXXXXXXXXXXXXXX
my blocking time measurment : 45
XXXXXXXXXXXXXXXXXXXXXXXXX
my non blocking time measurment : 0
buffer is empty after: 28

It is the flush() that hangs you up, of course.

So you have to make a function that watches the buffer, and call that in your loop() to see if the message has exited the building, not to wait for it to do, and your loop() will be back to running free.

I am unable yet to explain the 45 ms vs. 28 millisecond reports. It seems to be an indictment of the serial object flush().

Some casual experiments are intriguing, but my beach buddy has texted and she must not be kept waiting.

Be careful what is in (and out) of the measurement period. I got off track forgetting that printing any additional information will take time, and such time must be without the begin/end collection of time stamps.

a7

Software serial can do better than 9600. Not 115200, but you can jack up the speed somewhat.

Yes that is true. I knew this, I was unclear in my explanation. Knowing what flush() does I was sure what the outcome would be of the blocking variety. I expected the non-blocking variant to be precisely as long and that was my test. So I was expecting the same numbers (though I cannot acount for a 1ms discrepancy)

Meanwhile, the blocking and non-blocking essentials in this wokwi

Link to basicBlockNoBLock

seems to show that flush() takes way longer than waiting for the buffer to empty by repeatedly calling availableForWrite().

If anything, I would have thought the opposite. A look at the source code for both went past the time I had allowed, I reached no conclusion.

// https://forum.arduino.cc/t/sscanf-bug-what-i-am-doing-wrong/1046254

uint32_t myBeginTime;
uint32_t myEndTime;
uint32_t anotherBeginTime;
uint32_t anotherEndTime;
uint32_t bufferEmptyTime;

void setup()
{
 Serial.begin(9600);
 Serial.println("block no block");

//  for (int nn = 10; nn <= 100; nn += 10)
//    testBlockNoBlock(nn);

  testBlockNoBlock(30);
}

void testBlockNoBlock(int nChars)
{

Serial.print("test with ");
Serial.println(nChars);
Serial.println();

myBeginTime = micros();

  for (int i = 0; i < nChars; i ++)
  {
      Serial.write('X');     // send nChars X
  }
  Serial.println();

myEndTime = micros();

anotherBeginTime = micros();

  Serial.flush();

anotherEndTime = micros();

  int nAvailable = Serial.availableForWrite();

  Serial.print(nChars);
  Serial.print(" blocking time measurment : ");
  Serial.println(myEndTime - myBeginTime); 

  Serial.print(nChars);
  Serial.print(" flush() : ");
  Serial.println(anotherEndTime - anotherBeginTime); 

// - just reports 63. I guess the buffer is never really empty, haha.
//   Serial.print("          after serial flush,  availableForWrite() = ");
//   Serial.println(nAvailable);

  Serial.println();
  delay(777);  // why or why not, that is the question.

myBeginTime = micros();

  for (int i = 0; i < nChars; i ++)
  {
      Serial.write('X');     // send nChars X
  }
  Serial.println();

myEndTime = micros();

anotherBeginTime = micros();

 while (Serial.availableForWrite() != 63);  // just hanging out

anotherEndTime = micros();

  Serial.print(nChars);
  Serial.print(" non-blocking time measurment : ");
  Serial.println(myEndTime - myBeginTime); 

  Serial.print(nChars);
  Serial.print(" waiting for buffer to empty : ");
  Serial.println(anotherEndTime - anotherBeginTime); 
  Serial.println("");
}

void loop() { }


Results ready in seconds:
test with 30

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
30 blocking time measurment : 192
30 flush() : 66308

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
30 non-blocking time measurment : 200
30 waiting for buffer to empty : 32180

Both get the characters into the buffer rapidly. The *flush*() took 66 ms, waiting for the empty buffer took just 32.

Isn't a flushed buffer the same thing as one that has room for 63 characters?

a7