Is Serial1.available() useful for slow responding devices?

What else does your code have to do? I don't think anybody with experience will use delays in a while loop. And readBytes might timeout before the reply comes.

Can you provide details of the protocol? Exactly what do you send? What is a typical reply? CarriageReturn or Newline involved? Binary data or text?

Typical example will be useful.

@walshrd
The tutorial I mentioned in reply #15 is this one:

The solution to your problem lies in understanding and implementing the ideas in that tutorial, as I and others have pointed out, plus the ideas in Serial input basics as suggested by @J-M-L in reply #4. If those concepts are too advanced for you then step back and work on something simpler until you have learned enough to understand the content of those tutorials.

Your repeated questions suggest to me that you want an answer you are not going to get, that you don't like the answers you are being given so want a different answer. You have enough information to at least start working on a solution and to learn a great deal in the process.

That's always the case when your code doesn't have a stay-and-wait function blocking any other code from executing.
You want to run Serial fast so that it arrives and prints fast, clearing the output buffer quicker, the output buffer you never want to overfill.
Each serial char takes 1 start bit, 8 data bits, 1 stop bit. 9600 baud is 960 cps, 1042 micros per char. 115200 baud is 11,520 cps, 87 micros per char where a fast void loop runs 50 or less on average, closer to 20. 20 micros is 320 cpu cycles.

Below is non-blocking, event-driven pseudocode showing how
3 different things can run independently, together. Call them tasks.
Tasks can pass/share data in variables.

void loop()
{
if ( Serial.available() > 0 ) // only runs when a char is ready to read.
{
// read the char and process it. link below shows more than 1 way
// Gammon Forum : Electronics : Microprocessors : How to process incoming serial data without blocking
}

if ( millis() - startMs >= waitMs ) // watch the clock without any block
{
// do the timeout thing
// reset start time
}

if ( digitalRead( sensePin ) != senseLast ) // watch a pin for change
{
// do something
}
}

Thanx Sterretje. The Arduino sends "binary commands" in a one byte hex format: Serial1.write(buf, len); buf is a byte buffer with buf[0] = hex command (say 0x9D) and succeeding buf locations contain data required by the command: <data_1> <data_2> ...
The "device" should always send something back, whether it's data bytes or an acknowledgement receipt such as "!".

In my (very restricted) application, I will mostly only send the command byte plus a byte address (hex within the device memory) where the "device" has to read a byte and send that info (1 byte) back...sometimes adding another byte containing a "!" to indicate things went ok. Then sometimes only the requested data bytes are sent with no CR/LF. Some responses for some commands might include a CR/LF. Go figure. So I count the incoming bytes, and when Ive got the number Im promised, the rest are dumped or ignored. (Perhaps they shouldnt be ignored and always read and then dumped? --> That is a question.)

If the device is generally "out" servicing a ton of RS485 interrupts (that is its ordained function), the priority of servicing the RS232 port seems to be of lower priority much of the time. Everything operates at 9600 baud. Nothing better can be tolerated by the device. All this is why Im thinking: simple code for a simple result. Except for the possible issue of handling "foot dragging" on the part of the device.

I originally thought that a Serial1.available loop with delay()s might be ok, but you have reinforced a notion that this might not be a good option. Now Im once again back to Serial1.readBytes OR... per your thoughts (thanx in advance).

Then... perhaps I should throttle the device and send the same command rapidfire until a response occurs... But I know it is likely to get pissy and rapidfire me back with the same data over and over. A tit for tat thing. (Ive done this!)

Ok... I have read whatever there is in the now infamous tutorial which Im told I didnt read or understand. I hate to
say it, but Example 1 is of no value TO ME. Running an infinite loop at any level within void loop() to check status is not an option. I have read this
long tut several times, and brain fog has not appeared. What if the sending device does NOT respond, or takes a lot of time? Hmmmm. Or its responses do not have well-defined terminations? Those are reasons I came to the Forum with my questions. I didnt want the whole enchilada.

Sterretje seems to understand what Im dealing with: an old device that acts old and cranky at times (not always) with RS232.

My question has always been about the necessity to run a finite, short loop using Serial1.available WITHIN void loop() OR use Serial1.readBytes (no internal looping) OR something else that Forum experts would deem a better choice. That seems to have caused an uproar.

PeterBebbington stays above the fray handing out advice that got/gets me nowhere. So, Im advised to return to more simple, untaxing designs until my expertise/experience is deemed of sufficient level to return to the serious realm of Serial. He's not coming down to my low level to address really dirt simple questions about what is a best approach to handle old and cranky's behavior. I know about Serial, but I get curve balls thrown at me from this old RS232 device because it's sometimes loaded up servicing a very high-priority RS485 bus that it cares much more about. That seems to be lost on Peter...or never found him.

Im sure someone looking down here has the royal power to erase the totality of this unfortunate exchange. Please do so.

therefore you got the hint to use a state machine.
Following the answer #13 you can react on a timout in the state

RCVREPLY, // wait for complete reply

You can store the current timestamp/millis when you have sent a message to your device in a variable and compare it over and over again during the state RCREPLY for a timeout. If you don't get a full answer from your device --> handle the timeout. This could be either a new state for the sending device or just the state COMPLETE. It dependes what you want to do in that case.

if you don't receive well defined terminations, you might run in a timeout OR you recognize invald terminations and use that as exit event for the state. In both cases you are in well defined states of your state machine.

It's a tutorial. Just "reading" might not be enough. Try the examples.

If you want a solution to your problem you either

  • will need to understand the serial input basics and the concept of state machines
  • or you will need to get one to make the job done for you

Just in case you need some further help with this forum:

  • don't blame others that they haven't understood your question.
    Blame yourself for not describing your situation better. Do it better the next time.
  • If you don't understand an answer - ask.
3 Likes

The tutorial was recommended based on the amount of detail that you initially provided.

So here is a possible setup intended for demo purposes; extract whatever you need. The program has two pre-cooked queries in memory and using serial monitor you can pick one of them; the way it's set up allows for a total of 10 pre-cooked commands number 0 .. 9. The pre-cooked commands are shown below; adjust them to your needs.

// queries for the device
const uint8_t query1[] = {0x33, 0};
const uint8_t query2[] = {0x30, 2, 0x41, 0x42};

query1 and query2 are poorly choosen names; the names should be descriptive, e.g. resetDevice, setBaudrate, getStatus; I leave that up to your creativity.

Now every pre-cooked command has an expected length for the reply; it possibly might also have different duration before a reply is received. We can combine this all in a struct or class. Still being more of a C programmer than a C++ programmer, I will use a struct. A struct is like an entry in a phone book with a name, a phone number and possibly some other information. You can define a struct as shown below

// struct combining relevant information for communication with device
struct CMD
{
  const uint8_t *cmd;     // query (from above)
  const uint8_t size;     // number of bytes in query
  const uint8_t expected; // number of expected bytes in reply;
  const uint32_t timeout; // timeout value
};

The struct has a pointer to the query, the size of the query (total number of bytes (2 and 4 respectively), the number of expected bytes in the reply and a value for the timeout; other fields can be added. You can create an array of those structs as shown below.

const CMD commands[] =
{
  {query1, NUMELEMENTS(query1), 10, defaultTimeout},
  {query2, NUMELEMENTS(query2),  3, defaultTimeout},
};

NUMELEMENTS and defaultTimeout are defined elsewhere. You can replace defaultTimeout by a value (e.g. 500).

// number of elemenets in any array
#define NUMELEMENTS(x) (sizeof(x) / sizeof(x[0]))
...
...
// default timeout in ms; 0 equals no timeout (wait forever)
const uint32_t defaultTimeout = 5000;

I've introduced you to the enum without much detail. It's basically a new type (like int, byte etc) with a couple of predefined values.

// enum for communication state nachine
enum COMMSTATE
{
  CS_SENDCMD,  // send the command
  CS_RCVREPLY, // wait for complete reply
  CS_PROCESS,  // process reply
  CS_COMPLETE, // done
};

setup() can now be something like below.

void setup()
{
  // setup serial with PC and device
  Serial.begin(57600);
  Serial1.begin(9600);

  // setup builtin led
  pinMode(pinLed, OUTPUT);

  // print the available commands
  printCommands();
  Serial.println(F("======"));

  // first state for state machine is CS_COMPLETE; inform user here
  Serial.println(F("commState = CS_COMPLETE"));
}

There is an additional led that will blink while everything is running, hence the pinMode. printCommands() will print the configured commands and is shown in the final code.

And loop() can look like below.

void loop()
{
  // run the state machine
  deviceStatemachine();
  // blink a led
  heartBeat();
}

It demonstrates some simple multitasking; the statemachine that will take care of receiving data from the device and blinking a led.

We can now implement the state machine for the communication with the device

/*
  implementation of state machine for communication with device and processing of device data
*/
void deviceStatemachine()
{
  // state machine state
  static COMMSTATE commState = CS_COMPLETE;
  // command from serial input (not Serial)
  static uint8_t selectedCommand;
  // start time of receive
  static uint32_t rcvStartTime;
  // reply status
  REPLYSTATUS rs;

  switch (commState)
  {
    case CS_COMPLETE:
      // wait for data from serial monitor to start the steps
      if (Serial.available() > 0)
      {
        // read number from serial monitor
        selectedCommand = Serial.read();
        Serial.print(F("Received: '"));
        Serial.write(selectedCommand);
        Serial.println(F("'"));
        // convert char to number
        selectedCommand -= '0';
        // check if number is invalid
        if (selectedCommand >= NUMELEMENTS(commands))
        {
          Serial.print(F("Invalid command selected"));
          // stay in current state waiting for a command
        }
        // number is valid
        else
        {
          // switch to send state
          commState = CS_SENDCMD;
          Serial.println(F("commState = CS_SENDCMD"));
        }
      }
      break;
    case CS_SENDCMD:
      // reset rcvFixedLength
      rcvFixedLength(commands[selectedCommand].expected, true);
      // send the command
      Serial1.write(commands[selectedCommand].cmd, commands[selectedCommand].size);
      // set the start time of the receive
      rcvStartTime = millis();
      // switch to receive state
      commState = CS_RCVREPLY;
      Serial.println(F("commState = CS_RCVREPLY"));
      break;
    case CS_RCVREPLY:
      // if there is a timeout
      if (millis() - rcvStartTime > commands[selectedCommand].timeout)
      {
        Serial.println(F("Timeout"));
        // switch to competed state
        commState = CS_COMPLETE;
        Serial.println(F("commState = CS_COMPLETE"));
      }
      // no timeout
      else
      {
        // check if data available and add to buffer
        rs = rcvFixedLength(commands[selectedCommand].expected, false);
        // if full reply received
        if (rs == RCV_COMPLETE)
        {
          // switch to process state
          commState = CS_PROCESS;
          Serial.println(F("commState = CS_PROCESS"));
        }
        else
        {
          // stay where we are, waiting for more bytes
        }
      }
      break;
    case CS_PROCESS:
      // process the reply up to expeced number of bytes
      processReply(commands[selectedCommand].expected);
      // switch to competed state
      commState = CS_COMPLETE;
      Serial.println(F("commState = CS_COMPLETE"));
      break;
  }
}

The code starts with the declaration of some variables; next a switch/case implements that actual state machine and the swithing between states. It starts in the state CS_COMPLETE where it waits for input from the serial monitor. Only if the user did send a command and the number is a valid index in the array of CMD structs, the code will switch to the next state wher it will send the command to the device.

CS_SENDCMD is quite straight forward. Just in case we had a timeout situation where a reply was partially received and the receive buffer contains old data, we first reset the internals of the rcvFixedLength() function. Next we send the command, remember the time when we start receiving and switch to the next state.

CS_RCVREPLY first checks if there is a timeout condition; if not, it will check for serial data using the function rcvFixedLength(). Once rcvFixedLength() indicates that the full message is received, the state machine will go to the next state where it will process the received data,

CS_PROCESS processes the reply and next change the state to CS_COMPLETE and the process starts from the beginning.

On any error, an error message will be shown on the serial monitor and the state will switch to CS_COMPLETE and the process starts from the beginning again.

rcvFixedLength() handles the reception of the device's reply. It takes two arguments, the expected length and a flag to reset the internals (just the index variable).

/*
  receive a specified number of bytes from device
  In:
    expected number of bytes in reply
    flag to reset index
*/
REPLYSTATUS rcvFixedLength(uint8_t expectedLength, bool reset)
{
  // index in receive buffer
  static uint8_t index;

  // reset the index so next time we start with a clean slate
  if (reset == true)
  {
    index = 0;
    return RCV_COMPLETE;
  }

  // if no bytes received (yet)
  if (index == 0)
  {
    // clear the buffer
    memset(bufferSerial, 0, NUMELEMENTS(bufferSerial));
  }

  // while there is data to be read, store it in the buffer
  while (Serial1.available() > 0)
  {
    // if we havent't received the full reply and the buffer is not yet full
    if (index != expectedLength && index < MAXREPLY)
    {
      bufferSerial[index] = Serial1.read();
      index++;
    }
  }

  // if expected number of bytes received
  if (index >= expectedLength)
  {
    // reset index so we don't continue appending
    index = 0;
    // indicate to caller that all bytes are received
    return RCV_COMPLETE;
  }

  // indicate to caller that receive is still in progress
  return RCV_INPROGRESS;
}

The while-loop will copy received data to the buffer; note that not all data has to be received in one shot; we might receive e.g. 9 bytes out of 10 and next the 10[sup]th[/sup] byte. What Serial Input Basics tried to show you was to handle that and here the same approach is used. You will have to call this function repeatedly to collect the complete reply from the device.

The buffer is declared as a global variable and has a size of MAXREPLY; you can adjust MAXREPLY to your needs.

The function returns one of the following from the below enum. I could have used a bool (false for in progress, true for complete) but this is a leftover from experimentation.

// enum for reply status
enum REPLYSTATUS
{
  RCV_INPROGRESS, // not all expected bytes received yet
  RCV_COMPLETE,   // expected number of bytes received
};

Below is a simple example for the processing of the received data that prints the rceived data in HEX representation. It takes one parameter, the (expected) length.

/*
  example code for processing of received device data
  simply prints the received data in HEX representation
  In:
    length of received data
*/
void processReply(uint8_t length)
{
  for (uint8_t dataCnt = 0; dataCnt < length; dataCnt++)
  {
    if (bufferSerial[dataCnt] < 0x10)
    {
      Serial.print("0x0");
    }
    else
    {
      Serial.print("0x");
    }
    Serial.print(bufferSerial[dataCnt], HEX);
    Serial.print(F("\t"));
  }
  Serial.println();
}

Lastly we have the heartBeat function that simply blinks the led. If you use this function in combination with the simple readBytes, you will notice that while readBytes is in progress, it will affect the blinking rate.

/*
  heartBeat blinks the builtin led in a non-blocking way
*/
void heartBeat()
{
  static uint32_t nextTime;
  static uint8_t ledState = LOW;

  if (millis() > nextTime)
  {
    nextTime += 250;
    ledState = !ledState;
    digitalWrite(pinLed, ledState);
  }
}

The full code

// number of elemenets in any array
#define NUMELEMENTS(x) (sizeof(x) / sizeof(x[0]))
// max reply size; adjust to needs
#define MAXREPLY 32

const byte pinLed = LED_BUILTIN;

// receive buffer
uint8_t bufferSerial[MAXREPLY];
// default timeout in ms; 0 equals no timeout (wait forever)
const uint32_t defaultTimeout = 5000;

// enum for reply status
enum REPLYSTATUS
{
  RCV_INPROGRESS, // not all expected bytes received yet
  RCV_COMPLETE,   // expected number of bytes received
};

// enum for communication state nachine
enum COMMSTATE
{
  CS_SENDCMD,  // send the command
  CS_RCVREPLY, // wait for complete reply
  CS_PROCESS,  // process reply
  CS_COMPLETE, // done
};

// queries for the device
const uint8_t query1[] = {0x33, 0};
const uint8_t query2[] = {0x30, 2, 0x41, 0x42};

// struct combining relevant information for communication with device
struct CMD
{
  const uint8_t *cmd;     // query (from above)
  const uint8_t size;     // number of bytes in query
  const uint8_t expected; // number of expected bytes in reply;
  const uint32_t timeout; // timeout value
};

const CMD commands[] =
{
  {query1, NUMELEMENTS(query1), 10, defaultTimeout},
  {query2, NUMELEMENTS(query2),  32, defaultTimeout},
};

void setup()
{
  // setup serial with PC and device
  Serial.begin(57600);
  Serial1.begin(9600);

  // setup builtin led
  pinMode(pinLed, OUTPUT);

  // print the available commands
  printCommands();
  Serial.println(F("======"));

  // first state for state machine is CS_COMPLETE; inform user here
  Serial.println(F("commState = CS_COMPLETE"));
}

void loop()
{
  // run the state machine
  deviceStatemachine();
  // blink a led
  heartBeat();
}

/*
  implementation of state machine for communication with device and processing of device data
*/
void deviceStatemachine()
{
  // state machine state
  static COMMSTATE commState = CS_COMPLETE;
  // command from serial input (not Serial)
  static uint8_t selectedCommand;
  // start time of receive
  static uint32_t rcvStartTime;
  // reply status
  REPLYSTATUS rs;

  switch (commState)
  {
    case CS_COMPLETE:
      // wait for data from serial monitor to start the steps
      if (Serial.available() > 0)
      {
        // read number from serial monitor
        selectedCommand = Serial.read();
        Serial.print(F("Received: '"));
        Serial.write(selectedCommand);
        Serial.println(F("'"));
        // convert char to number
        selectedCommand -= '0';
        // check if number is valid
        if (selectedCommand >= NUMELEMENTS(commands))
        {
          Serial.print(F("Invalid command selected"));
          // switch to competed state
          commState = CS_COMPLETE;
          Serial.println(F("commState = CS_COMPLETE"));
        }
        else
        {
          // switch to send state
          commState = CS_SENDCMD;
          Serial.println(F("commState = CS_SENDCMD"));
        }
      }
      break;
    case CS_SENDCMD:
      // reset rcvFixedLength
      rcvFixedLength(commands[selectedCommand].expected, true);
      // send the command
      Serial1.write(commands[selectedCommand].cmd, commands[selectedCommand].size);
      // set the start time of the receive
      rcvStartTime = millis();
      // switch to receive state
      commState = CS_RCVREPLY;
      Serial.println(F("commState = CS_RCVREPLY"));
      break;
    case CS_RCVREPLY:
      // if there is a timeout
      if (millis() - rcvStartTime > commands[selectedCommand].timeout)
      {
        Serial.println(F("Timeout"));
        // switch to competed state
        commState = CS_COMPLETE;
        Serial.println(F("commState = CS_COMPLETE"));
      }
      // no timeout
      else
      {
        // check if data available and add to buffer
        rs = rcvFixedLength(commands[selectedCommand].expected, false);
        // if full reply received
        if (rs == RCV_COMPLETE)
        {
          // switch to process state
          commState = CS_PROCESS;
          Serial.println(F("commState = CS_PROCESS"));
        }
        else
        {
          // stay where we are, waiting for more bytes
        }
      }
      break;
    case CS_PROCESS:
      // process the reply up to expeced number of bytes
      processReply(commands[selectedCommand].expected);
      // switch to competed state
      commState = CS_COMPLETE;
      Serial.println(F("commState = CS_COMPLETE"));
      break;
  }
}

/*
  receive a specified number of bytes from device
  In:
    expected number of bytes in reply
    flag to reset index
*/
REPLYSTATUS rcvFixedLength(uint8_t expectedLength, bool reset)
{
  // index in receive buffer
  static uint8_t index;

  // reset the index so next time we start with a clean slate
  if (reset == true)
  {
    index = 0;
    return RCV_COMPLETE;
  }

  // if no bytes received (yet)
  if (index == 0)
  {
    // clear the buffer
    memset(bufferSerial, 0, NUMELEMENTS(bufferSerial));
  }

  // while there is data to be read, store it in the buffer
  while (Serial1.available() > 0)
  {
    // if we havent't received the full reply and the buffer is not yet full
    if (index != expectedLength && index < MAXREPLY)
    {
      bufferSerial[index] = Serial1.read();
      index++;
    }
  }

  // if expected number of bytes received
  if (index >= expectedLength)
  {
    // reset index so we don't continue appending
    index = 0;
    // indicate to caller that all bytes are received
    return RCV_COMPLETE;
  }

  // indicate to caller that receive is still in progress
  return RCV_INPROGRESS;
}

/*
  example code for processing of received device data
  simply prints the received data in HEX representation
  In:
    length of received data
*/
void processReply(uint8_t length)
{
  for (uint8_t dataCnt = 0; dataCnt < length; dataCnt++)
  {
    if (bufferSerial[dataCnt] < 0x10)
    {
      Serial.print("0x0");
    }
    else
    {
      Serial.print("0x");
    }
    Serial.print(bufferSerial[dataCnt], HEX);
    Serial.print(F("\t"));
  }
  Serial.println();
}

/*
  heartBeat blinks the builtin led in a non-blocking way
*/
void heartBeat()
{
  static uint32_t nextTime;
  static uint8_t ledState = LOW;

  if (millis() > nextTime)
  {
    nextTime += 250;
    ledState = !ledState;
    digitalWrite(pinLed, ledState);
  }
}

/*
  print the available queries for the device and xpected number of bytes in reply
  Note:
    the function will hang forever if you specify an expected length that exceeds the buffer size
*/
void printCommands()
{
  for (uint8_t cmdCnt = 0; cmdCnt < NUMELEMENTS(commands); cmdCnt++)
  {
    Serial.print(F("Command "));
    Serial.println(cmdCnt + 1);
    for (uint8_t byteCnt = 0; byteCnt < commands[cmdCnt].size; byteCnt++)
    {
      if (commands[cmdCnt].cmd[byteCnt] < 0x10)
      {
        Serial.print("0x0");
      }
      else
      {
        Serial.print("0x");
      }
      Serial.print(commands[cmdCnt].cmd[byteCnt], HEX);
      Serial.print(F("\t"));
    }
    Serial.println();
    Serial.print(F("Expected reply size = "));
    Serial.println(commands[cmdCnt].expected);
    if (commands[cmdCnt].expected > MAXREPLY)
    {
      Serial.println(F("Expected reply size exceeds buffer size"));
      for (;;);
    }
  }
}

I hope this is sufficient information to give you the idea. As I don't have further insight in your application, this is the best I can do as a demo. It's basically tested with only Serial and not Serial1.

1 Like

Sterretje: This is a tutorial for the ages! I sincerely THANK YOU! There is a lot to learn here, and
I have already begun studying your code. I have sliced out a section here and there and ran those as
standalones (with a few additions) to actually see what happens. It's very instructive for me. Thank
you again for all your effort on my behalf. It's greatly appreciated.