Trying to understand Serial.read()

Projects Programming
1

I have made a sketch for the ESP8266 that reads my electricity meter through an MBUS to ttl converter.
The program works fine, but I would like to understand the serial read process better than I do. So I made a small sketch that shows how data is read, and counts the number of times the loop calls the read function in the sketch before there is a message present.

The meter only outputs at 2400 bd, but the serial buffer is checked (that's how I understand it anyway) many times before each complete message. (Serial.available() > 0) is a condition to start reading.

The telegrams are 41 and 123 bytes long respectively, and seems complete every time they are read.

What puzzles me is: Every time data is reported, it seems to be a complete message and not parts of a message. I would expect that many times only part of a message would be read, because the baudrate is low and the loop seems to run very fast.

Could someone please explain why only complete messages are reported, and not parts of messages?

Code:

#include <SoftwareSerial.h>

SoftwareSerial mySerial(12, 14); // RX, TX

unsigned char buf[4096];
int buf_len = 0;
boolean newData = false;
int readCount = 0;


void setup() {
  // Wait 1 second
  delay(1000);
  
  // Serial reads the meter
  Serial.begin(2400, SERIAL_8E1);
  // mySerial prints to the monitor
  mySerial.begin(57600);
  
  Serial.swap();
}


void loop() {
  readDataTelegram();
  if (newData == true) {
    mySerial.print("readDataTelegram() run: ");
    mySerial.print(readCount + 1);
    mySerial.println(" times");
    readCount = 0;
    mySerial.print("Telegram just read with length (buf_len): ");
    mySerial.print(buf_len);
    mySerial.println(" bytes");
    mySerial.println();
    newData = false;
  } else {
    readCount++;
  }
}


// Read the serial buffer
void readDataTelegram() {
  static byte ndx = 0;
  char rc;
  while ((Serial.available() > 0) && (newData == false)) {
    rc = Serial.read();
    buf[ndx] = rc;
    ndx++;
  }
  if (ndx > 0) {
    buf_len = ndx;
    ndx = 0;
    newData = true;
  }
}

Result:

readDataTelegram() run: 244121 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244650 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 294617 times
Telegram just read with length (buf_len): 123 bytes

readDataTelegram() run: 191217 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244589 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244763 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244459 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 295225 times
Telegram just read with length (buf_len): 123 bytes

readDataTelegram() run: 191509 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244458 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244591 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 244813 times
Telegram just read with length (buf_len): 41 bytes

readDataTelegram() run: 292875 times
Telegram just read with length (buf_len): 123 bytes
2

What is the maximum value for an 'int' data type on a ESP8266?

3

What is the maximum value for an 'int' data type on a ESP8266?

1^31 - 1

Could someone please explain why only complete messages are reported, and not parts of messages?

On the ESP platform the Serial.available() call does not just return the number of bytes in the buffer but also calls optimistic_yield() to do some background tasks. The call is yielding only if the current loop() run is running for more than 10ms. The hardware has an internal buffer that is copied to the ring buffer in an interrupt. I don't know what exactly triggers this copying as I'm not an expert for this processor.

4

pylon:
1^31 - 1

0? LOL. (sorry couldn't resist)

That explains how readcount returns a such a large number (larger that 2^15-1).

5

adwsystems:
What is the maximum value for an 'int' data type on a ESP8266?

When ESP8266(NodeNCU1.0-12E Module) is being operated under Arduino IDE, should we shift from the understanding that the value of the int type variable can be from -32768 to 32767.

6

GolamMostafa:
When ESP8266(NodeNCU1.0-12E Module) is being operated under Arduino IDE, should we shift from the understanding that the value of the int type variable can be from -32768 to 32767.

Then how does read count exceed 32767 per the serial monitor output in the OP? Not using the Arduino IDE?

7

0? LOL. (sorry couldn't resist)

LOL. 2^31 - 1 is correct if someone didn't catch my error instantly.

When ESP8266(NodeNCU1.0-12E Module) is being operated under Arduino IDE, should we shift from the understanding that the value of the int type variable is from -32768 to 32767.

int is always platform specific. If you use a Due it's 32bit too. If you need a fixed size use the corresponding sized types (int16_t, int32_t, etc.).

8

pylon:
int is always platform specific. If you use a Due it's 32bit too. If you need a fixed size use the corresponding sized types (int16_t, int32_t, etc.).

I am beginning (forcing myself) to use those declarations in order to know (remind me) what size value I'm working with.

9

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data.

...R

10

pylon:
int is always platform specific. If you use a Due it's 32bit too. [...].

The execution of the following codes (under Arduino IDE + DUE) has established the quoted information.

void setup() 
{
  Serial.begin(9600);
  int x = 0x12345678;
  Serial.print(x, HEX); //shows: 12345678
}

void loop() 
{

}

Now, the following two declarations in the IDE+DUE Platform are the same. Which one to receive preference in DUE programming?

int x = 0x12345678;
long x = 0x12345678;
11

GolamMostafa:
The execution of the following codes (under Arduino IDE + DUE) has established the quoted information.

void setup()

{
 Serial.begin(9600);
 int x = 0x12345678;
 Serial.print(x, HEX); //shows: 12345678
}

void loop()
{

}

What happens if you run it on an UNO?

GolamMostafa:
Now, the following two declarations in the IDE+DUE Platform are the same. Which one to receive preference in DUE programming?

int x = 0x12345678;

long x = 0x12345678;

Odd way to ask. I would ask, on a DUE is a long 32 bit or 64 bit?

12

adwsystems:
What happens if you run it on an UNO?

Without executing the codes on the UNO, I can tell that the Serial Monitor would show: 5678?

13

GolamMostafa:
Without executing the codes on the UNO, I can tell that the Serial Monitor would show: 5678?

OK.

Do you know if on a DUE is a long 32 bit or 64 bit?

14

adwsystems:
Do you know if on a DUE is a long 32 bit or 64 bit?

The execution of the following codes on IDE+DUE Platform demonstrates that long refers to a 32-bit data size.

void setup()
{
  Serial.begin(9600);
  union myTag
  {
    long z;// long long z;
    int myData[2];
  } data;

  // data.z = 0x12345678ABCDEF12;
  data.z = 0x12345678ABCDEF12;

  Serial.println(data.myData[0], HEX); //shows: ABCDEF12
  Serial.println(data.myData[1], HEX); //shows: 0 (12345678)
}

void loop()
{

}
15

Sorry I took it a bit lightly on the readCount variable (post #1). I never thought the program would do that many loops between received messages.

Well, now I know that int is 32 bit on ESP8266.

Thanks for the input on Serial read. It helps.