SoftwareSerial read & write

I’m having an issue while reading and writing data using softwareserial. The idea is to read data from a device with frames that starts and ends with 0x5E, then write the data to another device.
But it seems that writing, somehow leads to corruption of incoming data. The serial.write() is for debugging. Ideally just the two first lines in loop() should be there. If I remove all serialD.write(), all data is received correct. The same is the case if I add some delay inside loop().
However this is just a small part of the final code, which will also read and process data from UART, before sending the results with serialD.write() (two inputs, one output).
Is there a way to make this work as expected?

#include <SoftwareSerial.h>

#define PIN_SerialTelemetryRX 10      
#define PIN_SerialTelemetryTX 12      
#define LED_PIN 13                     
#define MAX_FRAME 5

SoftwareSerial serialD(PIN_SerialTelemetryRX, PIN_SerialTelemetryTX, true);
uint8_t data;
byte count;

void processExternalByte (uint8_t ExternalByte) {
  static uint8_t DData[MAX_FRAME];
  static unsigned int index = 0;
  static byte stuffed = 0;
  
  if (ExternalByte == 0x5e) { // Head/Tail
    if (index == 3 + stuffed) { // Valid frame size
      digitalWrite(LED_PIN, HIGH);
      serialD.write((uint8_t)0x5e);
      Serial.print("\n");
      for (int i=0; i < index; i++) { 
        serialD.write(DData[i]);
        if (DData[i] < 0x10)
          Serial.print("0");
        Serial.print(DData[i], HEX);
      }
      serialD.write((uint8_t)0x5e);
      Serial.print("\n\n");
      digitalWrite(LED_PIN, LOW);
    }      
    index = 0;
    stuffed = 0;
    return;
  }
  if (index <  MAX_FRAME) { 
    DData[index++] = ExternalByte;
    if (ExternalByte == 0x5d)
      stuffed++;
  }    
}  

void setup()
{
  pinMode(LED_PIN, OUTPUT); 

  serialD.begin(9600); 
  Serial.begin(115200);
}

void loop() {
//  while(serialD.available() > 0)
//    processExternalByte(serialD.read());
 
  delay(50); // Works with this delay 
  do {
    count = serialD.available();
    if (count > 0) {
      data = serialD.read();
      if (data < 0x10) 
        Serial.print("0");
      Serial.print(data, HEX);
      processExternalByte(data); 
    }
  } while (count);   
}

Fred

Try receiving the data with the 3rd example in Serial Input Basics after changing the start- and end-markers to suit your requirement.

And study the concept by which it receives data without any requirement for a delay()

...R

flarssen:
Is there a way to make this work as expected?

Use a better UART emulation ibrary than SoftwareSerial.

I.e. "AltSoftSerial".

SoftwareSerial is half duplex; i.e., you can't read and write at the same time. When you do a write with SoftwareSerial it disables interrupts for a long time which in turn will block incoming data from being detected.

AltSoftSerial is full duplex.

Alternatively, and depending on the details of your data flow, you may be able to read and buffer the input and then write it later.

Thanks.
No wonder things gets messed up. I haven’t studied AltSoftSerial closely yet, but it seems that it doesn’t support inverse serial, nor the pins I’m using in the units i have made so far (Pro Mini based). I will try to tweak the reading/writing. Both data from UART and pin 10 will be written to pin 12, so I will try to synchronise this. like sending all data if no input data (fixed interval) has been available for some time, but the low speed (9600) doesn’t make this any easier.

That is the drawback with AltSoftSerial. You could modify it to handle inverted logic but the pins it uses are determined by the processor.

You might try the attached library and see if that works for you.

It’s a software serial implementation I wrote some time ago for my own use. I gave it to someone else to maintain (he calls it NeoSWSerial). But in the attached version I added a switch in the include file to allow interrupts during a write and set this switch appropriately. I also added an inverse logic option in the same form as SoftwareSerial uses.

I tested it with a simple sketch that checked for an available character, read it and then wrote it immediately. It worked fine at 9600 baud with a GPS as the source of characters. It might have problems in a system with other interrupts that steal too much time, but in a quiet system and at 9600 baud it looks like it can do full duplex. I did not test the inverse logic option however.

For what it’s worth.

gSoftSerial_noInterruptsWrite_27Jan2016.zip (5.38 KB)

My Yet Another Software Serial might be a basis from which you could develop your own code.

...R

Does any if these libraries support 8E1 setting?

Watcher:
Does any if these libraries support 8E1 setting?

How does that question help @flarssen ?

...R

jboyton:
That is the drawback with AltSoftSerial. You could modify it to handle inverted logic but the pins it uses are determined by the processor.

You might try the attached library and see if that works for you.

It's a software serial implementation I wrote some time ago for my own use. I gave it to someone else to maintain (he calls it NeoSWSerial). But in the attached version I added a switch in the include file to allow interrupts during a write and set this switch appropriately. I also added an inverse logic option in the same form as SoftwareSerial uses.

I tested it with a simple sketch that checked for an available character, read it and then wrote it immediately. It worked fine at 9600 baud with a GPS as the source of characters. It might have problems in a system with other interrupts that steal too much time, but in a quiet system and at 9600 baud it looks like it can do full duplex. I did not test the inverse logic option however.

For what it's worth.

Thanks!
I have to read 8 bit binary data. Do you think it would be possible to combine gSoftSerial.write() and SoftwareSerial.read()?

flarssen:
Do you think it would be possible to combine gSoftSerial.write() and SoftwareSerial.read()?

The SoftwareSerial receive interrupt routine disables interrupts for about 95% of the time data is incoming. So you can’t write at the same time.

flarssen:
I have to read 8 bit binary data.

I have so many versions of software serial. I’d forgotten that I have one based on timer 2 that receives binary data and sends the character with the interrupts enabled. I quickly edited it to add the (untested) inverse logic code and attached it.

Let me know if works.

sSoftSerial_27Jan2016.zip (3.75 KB)

jboyton:
I have so many versions of software serial. I’d forgotten that I have one based on timer 2 that receives binary data and sends the character with the interrupts enabled. I quickly edited it to add the (untested) inverse logic code and attached it.

Let me know if works.

OK. Did a small test, and did not receive the correct values. Changing the inverse flag made no difference.

SoftwareSerial data:
5E2800005E5E3A04005E5E3B05005E
5E2800005E5E3A04005E5E3B05005E
5E2800005E5E3A04005E5E3B05005E

sSoftSerial data:
00A8EDFF43438BF7FF434389F5FF43
00A8EDFF43438BF7FF434389F5FF43
00A8EDFF43438BF7FF434389F5FF43

//#include "SoftwareSerial.h"
#include "sSoftSerial.h"

#define PIN_SerialTelemetryRX 10       
#define PIN_SerialTelemetryTX 12   

sSoftSerial serialD(PIN_SerialTelemetryRX, PIN_SerialTelemetryTX, true);
//SoftwareSerial serialD(PIN_SerialTelemetryRX, PIN_SerialTelemetryTX, true);
uint32_t currentTime, lastDDataIn;
uint8_t data;
byte count;
boolean DDatareceived;

void setup()
{
  serialD.begin(9600);
  Serial.begin(115200);
}

void loop() {
  currentTime = millis();
  if (DDatareceived && currentTime - lastDDataIn > 150UL) {
    Serial.print("\n");
    DDatareceived = false;
  }

  do {
    count = serialD.available();
    if (count > 0) {
      data = serialD.read();
      if (data < 0x10)
        Serial.print("0");
      Serial.print(data, HEX);
      lastDDataIn = millis();
      if (DDatareceived == false)
        DDatareceived = true;
    }
  } while (count);
}

Fred

Sorry about that. I guess I did it wrong. It should have been an easy bit of coding but I didn’t take my time and look at it carefully, nor did I try to test it.

Let me take a quick look at the code.

edit:
Okay, I was pretty sloppy. There were two bugs: it was never actually getting set to inverse and one of the ISRs wasn’t modified to recognize it even if it were. I tested it this time with a simple loop back and it seems to work, at least when talking to itself.

sSoftSerial_28Jan2016.zip (3.81 KB)

Reading seems to work now, however it looks like I first get the last byte from the previous frame of data, but that could be my timing, which is only there for testing. I will take a closer look after work, with the real code that does the writing, etc.

5E5E2800005E5E3A04005E5E3B0200
5E5E2800005E5E3A04005E5E3B0200
5E5E2800005E5E3A04005E5E3B0200
5E5E2800005E5E3A04005E5E3B0200

I think we have a working solution here :wink:
I added writing and, also included GPS data from UART, and there was no loss of data as far as I could see.
Not sure if the data shift mentioned in my previous post causes a delay in data, though.

//#include "SoftwareSerial.h"
#include "sSoftSerial.h"
#include <TinyGPS.h>

#define PIN_SerialTelemetryRX 10       
#define PIN_SerialTelemetryTX 12   
#define LED_PIN 13            
#define MAX_FRAME 5

sSoftSerial serialD(PIN_SerialTelemetryRX, PIN_SerialTelemetryTX, true);
//SoftwareSerial serialD(PIN_SerialTelemetryRX, PIN_SerialTelemetryTX, true);
TinyGPS tinyGps;     // GPS object
uint32_t sentenceTime1, sentenceTime2;
uint32_t currentTime, lastDDataIn;
uint8_t data;
byte count;
byte gpsSentences = 0;
boolean DDatareceived;

void SendDValue(uint8_t ID, uint16_t Value) {
  uint8_t lsb = Value & 0x00ff;
  uint8_t msb = (Value & 0xff00)>>8;
  serialD.write(0x5E);
  serialD.write(ID);
  if(lsb == 0x5E) {
    serialD.write(0x5D);
    serialD.write(0x3E);
  }
  else if(lsb == 0x5D) {
    serialD.write(0x5D);
    serialD.write(0x3D);
  }
  else {
    serialD.write(lsb);
  }
  if(msb == 0x5E) {
    serialD.write(0x5D);
    serialD.write(0x3E);
  }
  else if(msb == 0x5D) {
    serialD.write(0x5D);
    serialD.write(0x3D);
  }
  else {
    serialD.write(msb);
  }
}

static void convertCoordinate(int32_t gpsCoordinate, uint16_t *b, uint16_t *a)
{
int32_t base, deg, min;
base = abs(gpsCoordinate); // base in micro degrees
deg = base / 1000000;
base = (base - deg * 1000000) * 60; // base in micro minutes
min = base / 1000000; // whole minutes left
*b = deg * 100 + min;
*a = (base - min * 1000000) / 100; // 4 digit minute decimal
}

void SendDGpsData(void) {
  int32_t lat, lon, alt;
  uint16_t msd, lsd, course, knots;
  int year;
  byte month, day, hour, minute, second, hundredths;  
  unsigned long age;
  uint16_t degr, mins;
 
  alt = tinyGps.altitude(); // +/- altitude in meters
  course = tinyGps.course(); // course in degrees
  knots = tinyGps.speed(); // speed in knots
  tinyGps.crack_datetime(&year, &month, &day, &hour, &minute, &second, &hundredths, &age);
  SendDValue(0x15, (uint16_t)(month << 8 | day));        
  SendDValue(0x16, (uint16_t)(year-2000));                      
  SendDValue(0x17, (uint16_t)(minute << 8 | hour));       
  SendDValue(0x18, (uint16_t)second);                    
  tinyGps.get_position(&lat, &lon, &age);
  convertCoordinate(lat, &msd, &lsd); 
  SendDValue(0x13, msd);        // Latitude (DDMM)
  SendDValue(0x1B, lsd);        // Latitude (.MMMM)
  SendDValue(0x23, (uint16_t)(lat < 0 ? 'S' : 'N'));
  convertCoordinate(lon, &msd, &lsd); 
  SendDValue(0x12, msd);        // Longitude (DDMM)
  SendDValue(0x1A, lsd);        // Longitude (.MMMM)
  SendDValue(0x22, (uint16_t)(lon < 0 ? 'W' : 'E'));
  SendDValue(0x01, (int16_t)(alt / 100));                   // Altitude m
  SendDValue(0x09, (uint16_t)(abs(alt) % 100));             // Altitude centimeter 
  SendDValue(0x11, (uint16_t)(knots / 100));                // Speed knots
  SendDValue(0x19, (uint16_t)(knots % 100));                // Speed decimals
  SendDValue(0x14, (uint16_t)(course / 100));               // Course degrees
  SendDValue(0x1C, (uint16_t)(course % 100));               // Course decimals
  serialD.write((uint8_t)0x5e);                       // terminate frame
}

void setup()
{
  serialD.begin(9600);
  Serial.begin(9600);
}

void processExternalByte (uint8_t ExternalByte) {
  static uint8_t DData[MAX_FRAME];
  static unsigned int index = 0;
  static byte stuffed = 0;
  
  if (ExternalByte == 0x5e) { // Head/Tail
    if (index == 3 + stuffed) { // Valid frame size
      digitalWrite(LED_PIN, HIGH);
      serialD.write((uint8_t)0x5e);
      for (int i=0; i < index; i++) {
        serialD.write(DData[i]);
      }
      serialD.write((uint8_t)0x5e);
      digitalWrite(LED_PIN, LOW);
    }      
    index = 0;
    stuffed = 0;
    return;
  }
  if (index <  MAX_FRAME) { 
    DData[index++] = ExternalByte;
    if (ExternalByte == 0x5d)
      stuffed++;
  }
}  

void loop() {
  while (Serial.available() > 0) {
    if (tinyGps.encode(Serial.read())) { // if a NMEA sentence is complete 
      if (!gpsSentences)  // if first of two sentences
        sentenceTime1 = millis();  // timestamp sentence 1
      else
        sentenceTime2 = millis();  // timestamp sentence 2
      gpsSentences++;
    }
    if (gpsSentences > 1) { // if more then one sentence
      if (sentenceTime2 - sentenceTime1 > 175) {// sentences not from same dataset, resync
        sentenceTime1 = sentenceTime2;
        gpsSentences = 1;
      }
      else { // process GPS data
        digitalWrite(LED_PIN, HIGH);
        SendDGpsData();
        digitalWrite(LED_PIN, LOW);
        gpsSentences = 0; 
      }
    }
  }
  while(serialD.available() > 0)
    processExternalByte(serialD.read());
}

Is it possible to combine this with SoftwareSerial? I get a compile error if I try; multiple definition of `__vector_3/4/5.

The thing is that my units uses another library that depends on SoftwareSerial when operating in a different mode. In this mode, it communicates on single wire on pin 12 at 57600 baud. During operation, it’s one or the other.

Fred

flarssen:
Is it possible to combine this with SoftwareSerial? I get a compile error if I try; multiple definition of `__vector_3/4/5.

The thing is that my units uses another library that depends on SoftwareSerial when operating in a different mode. In this mode, it communicates on single wire on pin 12 at 57600 baud. During operation, it's one or the other.

Both libraries use pin change interrupts and because of the way the Arduino environment is organized a library has to declare ISRs for all three of the interrupt banks, even though it only uses one.

You could edit the libraries to make it work (probably with a renamed copy of SoftwareSerial). If the two used RX pins in different pin change interrupt banks then it would just be a matter of commenting out some of the ISR declarations. If the pins are in the same bank it's more complicated because they'd have to share an ISR and determine which library each interrupt belonged to.

I have been running your code for a while, testing with different amounts of input data, and it performs really well. No data corruption, even with large amount of both Serial and sSoftSerial data. The limitation now seems to be receiving part connected to pin 12.

I tried to implement it in the library used for the other mode (see my last post). I have partly success when defining SSS_9600_TIMER_INCR as 8, but I guess this is not close enough to 57600 bd. Is there some other way to achieve both 9600 and 57600?

Thanks
Fred

I don't know if that timer 2 based approach could be made to work at 57.6k. The other one that uses timer 0 will work up to 38.4k so I suppose there's a chance. At that baud a bit comes every ~17us. I think the timer prescale would need to be changed and the initial compare value tweaked, maybe by experiment. I don't know if it would work or not.

Why not adapt the SoftwareSerial logic instead?

flarssen:
I have been running your code for a while,

As this follows @jboyton's Reply #15 I presume you are referring to his code.

...R

jboyton:
Why not adapt the SoftwareSerial logic instead?

I had a go some time ago, but was not able to get rid of the conflict during compilation and have a working solution at the same time.