Confusion about serial.read()

[der_frosti]

Hey everyone,
at the moment im using a Teensy LC Microcontroller to receive data from a jetson Xavier.
The Teensy is supposed to just receive the data and take actions according to that.
At the moment im just using serial.read() to get the data, but this sometimes stops working and also requires the data to be sent fast.
So i added a if(serial.available()) before the serial.read().
And this is where my confusion starts. When the if(serial.available()) my process for recognizing the data just stops working, wich seems odd.
So my question would be what exactly happens when i try to serial:read() without serial data present. And why dies the check for data available have an influence.
I sadly cant add the whole programm as an attachment. But here are the important lines. If i comment in the two lines in the loop i stops working. That meaning it still runs but the data is being read in a way that doesnt fit my structure anymore.

#define LUT_resolution 64
#define revolutions 4
#define clock_frequency 1000000
#define spiral_frequency 5000

#define laserpin 9
#define pausepin 10
#define ledpin 13

#define baud_rate 9600
#define message_length 8

#include <XY2_100.h>
XY2_100 galvo(clock_frequency);

//Prototypes
void spiral(int center, int radius);

//Data communication
char inByte=0;
char last_inByte=0;
int message_postion =0;
byte read_in[message_length] = {0};
bool do_laser=false;
bool do_calibration = false;
uint16_t Coordinate_x=0;
uint16_t Coordinate_y=0;
uint16_t Corner1_x=0;
uint16_t Corner1_y=0;
uint16_t Corner2_x=0;
uint16_t Corner2_y=0;
  
//Global Varaibles
const float pi = 3.14159265359;
float cos_lut[LUT_resolution];
float sin_lut[LUT_resolution];
int period_ms=0;

unsigned long last_time=0;
unsigned long time1=0,time2=0;

void setup() {
  galvo.begin();
  period_ms=int((1/(float)spiral_frequency)*1000);
  
  for(int i=0;i<LUT_resolution;i++){
    cos_lut[i]=cos(2*pi*( (float)i / LUT_resolution));
    sin_lut[i]=sin(2*pi*( (float)i / LUT_resolution));
  }
  pinMode(laserpin,OUTPUT);
  pinMode(pausepin,INPUT);
  pinMode(ledpin,OUTPUT);
  digitalWriteFast(ledpin,LOW);
  digitalWriteFast(laserpin,LOW);
  
  Serial.begin(baud_rate);
  while(!Serial){;}
  Serial.setTimeout(1000); 
  
}

void loop() {
  while(digitalRead(pausepin));

  //if(Serial.available()>0) {
    
    inByte = Serial.read();
     if ( inByte == 0xfe && (message_postion == 4 ) ){

       Coordinate_x=0;
       Coordinate_x |= read_in[0];
       Coordinate_x |= (read_in[1] << 8);
  
       Coordinate_y=0;
       Coordinate_y |= read_in[2];
       Coordinate_y |= (read_in[3] << 8);
       do_calibration=true;
       message_postion=0;
     }
     else if(inByte == 0xff && (message_postion == message_length)){
       Corner1_x=0;
       Corner1_x |= read_in[0];
       Corner1_x |= (read_in[1] << 8);
  
       Corner1_y=0;
       Corner1_y |= read_in[2];
       Corner1_y |= (read_in[3] << 8);
  
       Corner2_x=0;
       Corner2_x |= read_in[4];
       Corner2_x |= (read_in[5] << 8);
  
       Corner2_y=0;
       Corner2_y |= read_in[6];
       Corner2_y |= (read_in[7] << 8); 
       
       do_laser=true;
       message_postion=0;
     }
     else if (inByte != 0xff && inByte != 0xfe && message_postion < message_length ){
       read_in[message_postion] = inByte;
       message_postion++;
       
     }
     else{
       message_postion=0;
     }
  
   
  if(do_laser){
    
    int center_x=Corner1_x-((Corner1_x-Corner2_x)/2);
    int center_y=Corner1_y-((Corner1_y-Corner2_y)/2);
    int radius=min( abs(Corner2_x-Corner1_x) ,abs(Corner2_y-Corner1_y) );
    
    if(testmodus){galvo.setImageXY(Corner1_x, Corner1_y); delay(100);}
    else{spiral(center_x, center_y ,radius,3);}
    do_laser=false;
    do_calibration=false;
  }
  else if(do_calibration){

    galvo.setImageXY(Coordinate_x, Coordinate_y); 
    delay(10);
    digitalWriteFast(laserpin,HIGH);
  }
  //}
}

I would appreciate it if anyone has an idea of why this code behaves differently if check if there is data available. Also i would like to know if the serial timeout value has any impact in this program, or if it only used with other functions like serial.readBytes().

Thanks in advance

[groundFungus]

For serial input I can recommend the serial input basics tutorial. I has proven robust code examples for receiving serial data.

[red_car]

@der_frosti

just stops working

@der_frosti

And why dies

What do these mean exactly?

From reading your code, you expect 2 types of messages.

A calibration message 5 bytes long terminated with 0xFE.
A laser message 9 bytes long terminated with 0xFF.

Are you sure the data is being sent exactly how you expect?

[GolamMostafa]

@der_frosti

And this is where my confusion starts. When the if(serial.available()) my process for recognizing the data just stops working, wich seems odd.

You may practice/study the following simple sketch along with Fig-1 to understand the relation between available() and read() methods. The sketch receives data (in the form of ASCII Codes) from the InputBox of the Serial Monitor and then sends it back to the OutputBox of the Serial Monitor.

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  byte n = Serial.available(); //there is at least one char in Buffer
  if ( n != 0) //control goes to below as long as 1 character is  in Buffer  (edit)
  {
    char y = Serial.read(); //brings-out data from FIFO type Serial Buffer (Fig-1) 
    Serial.println(y);
  }
}

Figure-1: Serial Buffer

[noiasca]

@der_frosti

So my question would be what exactly happens when i try to serial:read() without serial data present.

according to the Arduino Reference

Serial.read() - Arduino Reference

you would receive -1 if no data is present.

The return value of read is an integer but you might loose information as you read your Serial in a char inByte;

-1 in integer is 0xffff. So if you just use one byte and have any checks on 0xff this might be an issue.

Contrary, if you do an check with Serial.available, you will never get a -1 from Serial.read.
For me this explains possible different behaviours of your unseen code.

In generaI support the hint from @groundFungus - follow the tutorial "Serial Input Basic", try the examples and they will give you an introduction to a reliable Serial communication.

[der_frosti]

I looked at your code an the picture and one thing that stands out is the second comment, which suggests that there have to be 2 Bytes(chars) in the buffer for it to be read. Am i understanding this correct?

[anon43666536]

@GolamMostafa

if ( n != 0) //control goes to below as long as 2 charcaters in Buffer
 

That comment is incorrect.
Control goes to the conditional block as soon as one character is available.

@noiasca

The return value of read is an integer

Almost - the return type of Serial.read is int

[GolamMostafa]

@anon43666536

That comment is incorrect.

Thanks for using very judgmental word - incorrect.

[GolamMostafa]

@anon43666536

Almost - the return type of Serial.read is int

But we say that the Serial Buffer is byte type; then, from where the upper byte is coming?

[GolamMostafa]

@der_frosti

I looked at your code an the picture and one thing that stands out is the second comment, which suggests that there have to be 2 Bytes(chars) in the buffer for it to be read. Am i understanding this correct?

My mistake has been corrected by @anon43666536 in post #7.

[anon43666536]

@GolamMostafa

we say that the Serial Buffer is byte.
type;

Who is "we"?
The Serial buffers (Rx and Tx) are specified as "unsigned char", but not "byte".

The Rx buffer contains received characters - they have no "upper byte"

@GolamMostafa

Thanks for using very judgmental word

You're welcome.

[der_frosti]

Ok i have looked at your answers and made a new version of my receive code based on the one tutorial.

void recvBytesWithStartEndMarkers() {
    boolean recvInProgress = false;
    byte ndx = 0;
    byte startMarker = 0xfd;
    byte endMarker1 = 0xfe;
    byte endMarker2 = 0xff;
    byte rb;
   

    while (Serial.available() > 0 && newData == false) {
        rb = Serial.read();

        if (recvInProgress == true) {
            if (rb != endMarker1 && rb != endMarker2 ) {
                receivedBytes[ndx] = rb;
                ndx++;
                digitalWriteFast(ledpin,HIGH);
                delay(20);
                digitalWriteFast(ledpin,LOW);
                delay(10);
                if (ndx >= numBytes) {
                    ndx = numBytes - 1;
                }
            }
            else{
                recvInProgress = false;
                ndx = 0;
                newData = true;
                if(rb == endMarker1){calibration_recv=true;}
                else if(rb == endMarker2){spiral_recv=true;}
            }
        }

        else if (rb == startMarker) {
            recvInProgress = true;
        }
    }
}

And it still doesnt work how i would i like to. But i figured stuff out. This code correctly differentiates between the two messages but the data in the message is wrong because of more more data byte beeing read than it should. I have confirmed that by setting the led pin to high once a normal data Byte is received and not a special marker and an oscilloscope to count the high peaks. I found that there is always one more data byte being read so 9 instead of 8 or 5 instead of 4. That explains why my old code wasnt working, cause i checked the message position of the endbyte as well.The question is now where the extra byte comes from and why it wasnt read without the if(serial.available()).
To send the data im using termios.h this is some of the send code.

	if(*connect_to_vernichter) this->vernichter->send(0xfd);
	if(*connect_to_vernichter) this->vernichter->send(this->cal_laser_pos_x2);
	if(*connect_to_vernichter) this->vernichter->send(this->cal_laser_pos_y2);
	if(*connect_to_vernichter) this->vernichter->send(0xfe); //end byte for calbration coords

The variables cal_laser_pos_x2,cal_laser_pos_y2 are of the data type uint16_t so 2 bytes.

void VernichterArduino::send(uint16_t word)
{
	this->arduino->send(&word, sizeof(word));
}

void Arduino::send(void* data, size_t size_bytes)
{
	if(write(this->serial_port, data, size_bytes) == -1) this->create_arduino_environment();
}

[noiasca]

can you post two examples from the serial output your device is really sending?
Displayed in HEX so wie can see really all bytes which get transmitted?

For example with "Coolterm" or another good serial program?

[GolamMostafa]

@anon43666536

Who is "we"?

As a non-native, I need to review English vocabulary to understand the scope of "we".

[der_frosti]

I would like to but i dont know how to connect the jetson xavier to my pc

[der_frosti]

So i solved the problem. After the startmarker is read the program reads another byte. I dont know where it comes from, but just reading that byte into a dummy variable solves the problem and everything works as intended. This is not an optimal solution and theoretically i should find the cause for this. As soon as i have enough time im going to implement all the Teensy functionality on the xavier this this fix is not going to be permanent.