When software serial RX is active, bytes are dropped on hardware serial RX

kolompos · October 18, 2019, 6:30pm

You can see a scope output attached.

The problem is consistent, but i don’t know why it happens.

Other factors:

the problem does not occur when the software serial pin is disconnected (code not changed)
the problem persists when the sw serial is not polled, but the wire is connected

The module on the sw serial bus is a GPS
It’s a GY-GPSV3-NEO M8M
The library is TinyGPSPlus

Tried directly and with level shifter.
Tried different pin.

Some snippets:
in loop:

  if(Serial.available())
  {
    delay(100);
    while(Serial.available())
    {
      helper = (char)Serial.read();
      #ifdef DEBUG_MODE
        Serial.print(helper);
      #endif
      command += helper;
      if(helper == EOL_CHAR)
      {
        command.remove(0,command.lastIndexOf('

also in loop:

  sw_serial_timer = millis();
  do 
  {
    while (ss.available())
    {
      gps.encode(ss.read());
    }
  } while (sw_serial_timer + SW_SERIAL_POLL_TIME > millis());

)); // delete up to ’


also in loop:

§DISCOURSE_HOISTED_CODE_1§


![Serial overlap error.PNG|2004x568](upload://dFPPRzLct4xZP327braCE0fJ48.png)
        #ifdef DEBUG_MODE
          Serial.print("COMMAND IS: ");
          Serial.println(command);
        #endif
        stepOne();
        command = "";
      }
    }

also in loop:

§_DISCOURSE_HOISTED_CODE_1_§

Robin2 · October 18, 2019, 9:23pm

kolompos:
Some snippets:

Snippets are a waste of time. Please post the shortest complete program that illustrates the problem.

In the meantime have a look at the examples in Serial Input Basics - simple reliable ways to receive data.

...R

kolompos · October 19, 2019, 12:34am

Almost the bare minimum that produces the error:

#include <EEPROM.h>
#include <TinyGPS++.h>
#include <SoftwareSerial.h>

#define DEBUG_MODE

#define EOL_CHAR                '\n'

#define HARTBEAT_LED            8
#define SW_RX_PIN               4
#define SW_TX_PIN               3

#define GPS_BAUD                 9600

#define HARTBEAT_PERIOD         5000
#define SW_SERIAL_POLL_TIME     1000

TinyGPSPlus gps;
SoftwareSerial ss(SW_RX_PIN, SW_TX_PIN);

String command;
uint32_t prew_time, sw_serial_timer;
char helper;

void setup()
{
  // pinmodes
  pinMode(HARTBEAT_LED, OUTPUT);
  pinMode(SW_RX_PIN, INPUT);
  pinMode(SW_TX_PIN, OUTPUT);
  
  Serial.begin(38400);
  ss.begin(GPS_BAUD);
  #ifdef DEBUG_MODE
    Serial.println("---!!!DEBUG MODE ACTIVATED!!!---");
  #endif
  
  //BOOT OK
  #ifdef DEBUG_MODE
    Serial.println("BOOT OK");
  #endif
  prew_time = millis();
}

void loop()
{
  if(Serial.available())
  {
    while(Serial.available())
    {
      helper = (char)Serial.read();
      #ifdef DEBUG_MODE
        Serial.print(helper);
      #endif
      command += helper;
      if(helper == EOL_CHAR)
      {
        command.remove(0,command.lastIndexOf('

Hope it gives more clue.

The minimal circuit to it is the GPS modul with the UNO. Connected to pin 3&4.)); // delete up to ’


Hope it gives more clue.

The minimal circuit to it is the GPS modul with the UNO. Connected to pin 3&4.

        command = "";
      }
    }
  }
  
  if(prew_time + HARTBEAT_PERIOD < millis())
  {
    prew_time = millis();
    
    digitalWrite(HARTBEAT_LED,!digitalRead(HARTBEAT_LED));
    #ifdef DEBUG_MODE
      Serial.println("HARTBEAT");
    #endif
    
    #ifdef ENABLE_SENSOR
      readSensors();
    #endif
  }

  sw_serial_timer = millis();
  do 
  {
    while (ss.available())
    {
      gps.encode(ss.read());
    }
  } while (sw_serial_timer + SW_SERIAL_POLL_TIME > millis());
}

Hope it gives more clue.

The minimal circuit to it is the GPS modul with the UNO. Connected to pin 3&4.

Robin2 · October 19, 2019, 7:31am

My wild guess is that this is the problem

do 
  {
} while (sw_serial_timer + SW_SERIAL_POLL_TIME > millis());

How long does that take to complete?

In general don't use FOR or WHILE unless their loop completes within a few microsecs. Otherwise you are blocking the Arduino just as surely as using delay()

You should always test mills() using subtraction to avoid problems when millis() rolls over. For example

if (millis() - sw_serial_timer >= SW_SERIAL_POLL_TIME) {

(how I hate typing underscores )

It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. This can happen after the program has been running perfectly for some time. Just use cstrings - char arrays terminated with '\0' (NULL).

...R

kolompos · October 19, 2019, 10:55am

In the first post I describe 2 cases that rule out that wild guess.

kolompos:
Other factors:

the problem does not occur when the software serial pin is disconnected (code not changed)

the problem persists when the sw serial is not polled, but the wire is connected

The latter means that the do while is /* */ -d out.

Is it possible that the hw serial is not entirely hw based and also need some interrupt?
Because sw serial surely needs and during sw serial's interrupt others can not happen.

Robin2 · October 19, 2019, 1:53pm

kolompos:
The latter means that the do while is /* */ -d out.

Please post that version of your program.

How long is a single message for {a} HardwareSerial and {b} SoftwareSerial?

...R

kolompos · October 20, 2019, 12:11am

#include <EEPROM.h>
#include <TinyGPS++.h>
#include <SoftwareSerial.h>

#define DEBUG_MODE

#define EOL_CHAR                '\n'

#define HARTBEAT_LED            8
#define SW_RX_PIN               4
#define SW_TX_PIN               3

#define GPS_BAUD                 9600

#define HARTBEAT_PERIOD         5000
#define SW_SERIAL_POLL_TIME     1000

TinyGPSPlus gps;
SoftwareSerial ss(SW_RX_PIN, SW_TX_PIN);

String command;
uint32_t prew_time, sw_serial_timer;
char helper;

void setup()
{
  // pinmodes
  pinMode(HARTBEAT_LED, OUTPUT);
  pinMode(SW_RX_PIN, INPUT);
  pinMode(SW_TX_PIN, OUTPUT);
  
  Serial.begin(38400);
  ss.begin(GPS_BAUD);
  #ifdef DEBUG_MODE
    Serial.println("---!!!DEBUG MODE ACTIVATED!!!---");
  #endif
  
  //BOOT OK
  #ifdef DEBUG_MODE
    Serial.println("BOOT OK");
  #endif
  prew_time = millis();
}

void loop()
{
  if(Serial.available())
  {
    while(Serial.available())
    {
      helper = (char)Serial.read();
      #ifdef DEBUG_MODE
        Serial.print(helper);
      #endif
      command += helper;
      if(helper == EOL_CHAR)
      {
        command.remove(0,command.lastIndexOf('

The hw serial messages are less than 10 char, the sw serials are GPS NEMA sentences, about 50 chars or so.)); // delete up to ’


The hw serial messages are less than 10 char, the sw serials are GPS NEMA sentences, about 50 chars or so.

        command = "";
      }
    }
  }
  
  if(prew_time + HARTBEAT_PERIOD < millis())
  {
    prew_time = millis();
    
    digitalWrite(HARTBEAT_LED,!digitalRead(HARTBEAT_LED));
    #ifdef DEBUG_MODE
      Serial.println("HARTBEAT");
    #endif
    
    #ifdef ENABLE_SENSOR
      readSensors();
    #endif
  }

/*
  sw_serial_timer = millis();
  do 
  {
    while (ss.available())
    {
      gps.encode(ss.read());
    }
  } while (sw_serial_timer + SW_SERIAL_POLL_TIME > millis());
*/
}

The hw serial messages are less than 10 char, the sw serials are GPS NEMA sentences, about 50 chars or so.

Robin2 · October 20, 2019, 9:25am

kolompos:
The hw serial messages are less than 10 char, the sw serials are GPS NEMA sentences, about 50 chars or so.

Am I correct in my understanding that with the code in Reply #6 you miss some HardwareSerial data when something is connected to Pin4 and you do not miss any data when nothing is connected to Pin4?

What are you connecting to Pin4 for this test?

I have rarely used SoftwareSerial and I have no idea what might be causing the problem. What is the interval between the 10-character messages?

Maybe try AltSoftSerial or NeoSWSerial?

...R

kolompos · October 20, 2019, 12:21pm

Robin2:
Am I correct in my understanding that with the code in Reply #6 you miss some HardwareSerial data when something is connected to Pin4 and you do not miss any data when nothing is connected to Pin4?

Yes, you are. Also, on the first post, in the attached image, I do not miss data when hit the uC in the pause of the GPS modul.

They are the NEMA sentences from the GPS modul. They are sent every second or so.

!Changed the sw serial library to the suggested NeoSWSerial one, and now it is fine!

I have no dropped char on the hw serial. I think it had sometging to do with the interrupts. I haven’t checked with scope, but before 80% of the messages were cut, now I can’t find one.

So… let’s call it SOLVED ? Although the mystery isn’t solved.

Robin2 · October 20, 2019, 2:05pm

kolompos:
So... let's call it SOLVED ? Although the mystery isn't solved.

The standard SoftwareSerial library is not great.

...R

system · May 6, 2021, 11:35am