Serial uart sometimes doesn't send the first two characters

[ageis]

My project is to enable me to control my burgular alarm panel over the internet. It uses the Arduino Uno to communicate with the alarm panel. The uno acts as a driver for the web interface which is a Node JS web application on a Raspberry Pi.

The serial communication to do this has already been written for me. It uses the software serial library since the uno only has one hardware uart.

I think there lies the root of my problem. I think it spends too many clock cycles receiving the data sent by the alarm panel and not enough handling the sending of the uart.

I get problems
like the following

sending D: Sunday 01 Jan 01:34 when it shud be LCD: Sunday 01 Jan 01:34
sending D:...........P when it shud be LED:...........P
sending + :38 shud be just + so it appears to be sending + then some remaining charcters from previous transmission

I have verified by turning off the server application for the web interface, that it's a problem with the Uno and not the Node JS application.

source code

[alto777]

You have linked to a zip of your code.

It would receive far more attention, any of mine for example, if you post it here.

First use the IDE Autoformat tool if your code isn't already pretty formatted nicely.

Next use the IDE Copy for Forum tool on the sketch, come here and paste it into your next post on this thread.

TIA

a7

[ageis]

@alto777

First use the IDE Autoformat tool if your code isn't already pretty formatted nicely.

@alto777

Copy for Forum tool

#include "HardwareSerial.h"
#include "Arduino.h"
/*
  AccentaG4.cpp - Library for Accenta G4 panel.
  Created by Luca Paolini, March 23, 2017.
*/

#include "AccentaG4.h"

AccentaG4::AccentaG4(uint8_t rxPin, uint8_t txPin, uint8_t setPin,
                     uint8_t abortPin, uint8_t intPin, uint8_t paPin,
                     void (*sendMessage)(String msg))
  : serial(rxPin, txPin) {
  this->setPin = setPin;
  this->abortPin = abortPin;
  this->intPin = intPin;
  this->paPin = paPin;
  this->sendMessage = sendMessage;
  pinMode(setPin, INPUT_PULLUP);
  pinMode(abortPin, INPUT_PULLUP);
  pinMode(intPin, INPUT_PULLUP);
  pinMode(paPin, INPUT_PULLUP);
}

void AccentaG4::begin() {
  serial.begin(BUS_SPEED);
  tx.start = millis();
  queryStatus();
}

void AccentaG4::end() {
  serial.end();
}

void AccentaG4::sendKey(char key) {

  switch (key) {
    case '0':
      tx.queue.push(K_0);
      break;
    case '1':
      tx.queue.push(K_1);
      break;
    case '2':
      tx.queue.push(K_2);
      break;
    case '3':
      tx.queue.push(K_3);
      break;
    case '4':
      tx.queue.push(K_4);
      break;
    case '5':
      tx.queue.push(K_5);
      break;
    case '6':
      tx.queue.push(K_6);
      break;
    case '7':
      tx.queue.push(K_7);
      break;
    case '8':
      tx.queue.push(K_8);
      break;
    case '9':
      tx.queue.push(K_9);
      break;
    case 'c':
      tx.queue.push(K_CHIME);
      break;
    case 'o':
      tx.queue.push(K_OMIT);
      break;
    case 'x':
      tx.queue.push(K_CANCEL);
      break;
    case 'p':
      tx.queue.push(K_PROGRAM);
      break;
    case 'v':
      tx.queue.push(K_CONFIRM);
      break;
    case 's':
      tx.queue.push(K_SELECT);
      break;
    case '!':
      tx.queue.push(K_SOS);
      break;
  }
}

void AccentaG4::readPanelSignals() {
  int signals;
  bitWrite(signals, 0, setPin && !digitalRead(setPin));
  bitWrite(signals, 1, abortPin && !digitalRead(abortPin));
  bitWrite(signals, 2, intPin && !digitalRead(intPin));
  bitWrite(signals, 3, paPin && !digitalRead(paPin));
  setPanelSignals(signals);
}

void AccentaG4::readBusMessages() {
  // read keypad messages
  int data;

  lastReadBus = millis();
  while (serial.available()) {
    data = serial.read();
    if (data & 0x100) {          // mark parity -> start of message
      rx.data[0] = data & 0xff;  // drop parity bit
      rx.ptr = 1;
    } else if (rx.ptr < MSG_MAXLEN) {
      switch (rx.data[0]) {
        case P_COMMAND:
          if (rx.ptr < 3) {
            // bytes 1 and 2: data
            rx.data[rx.ptr++] = data;
          } else if (rx.ptr == 3) {
            // byte 3: checksum
            if (validateChecksum(data)) {
              setLedStatus(rx);
              lastLCDUpdate = millis();
            }
            rx.ptr++;
          }
          break;
        case L_COMMAND:
          if (rx.ptr == 1 || rx.ptr < rx.data[1] + 2) {
            // byte 1: length
            // byte 2 to length + 1: data
            rx.data[rx.ptr++] = data;
          } else if (rx.ptr == rx.data[1] + 2) {
            // byte length + 2: checksum
            if (validateChecksum(data)) {
              rx.data[rx.ptr] = '\0';  // terminate string
              setLcdStatus(rx);
            }
            rx.ptr++;
          }
          break;
        default:
          break;  // ignore other messages
      }
    }
  }
}

void AccentaG4::setPanelSignals(int signals) {
  if (signals != status.signals) {
    status.signals = signals;
    sendPanelSignals();
  }
}

void AccentaG4::sendPanelSignals() {
  // SET, ABORT, INTRUDER, PANIC
  char value[] = PANEL_SIGNALS;
  for (int i = 0; i < 4; i++) {
    bitRead(status.signals, i) || (value[i] = '.');
  }
  sendMessage("SIG:" + String(value));
}

void AccentaG4::setLedStatus(struct Rx rx) {
  status.led = rx.data[2] << 8 | rx.data[1];
  sendLedStatus();
}

void AccentaG4::sendLedStatus() {
  // ZONE 1-8, UNSET, TAMPER, SOS, POWER
  char value[] = LED_STATUS;
  for (int i = 0; i < 12; i++) {
    bitRead(status.led, i) || (value[i] = '.');
  }
  sendMessage("LED:" + String(value));
}

void AccentaG4::setLcdStatus(struct Rx rx) {
  strncpy(status.lcd, rx.data + 2, rx.ptr - 1);
  sendLcdStatus();
}

void AccentaG4::sendLcdStatus() {
  long lastReadBusElapsed = millis() - lastLCDUpdate;

  if (lastReadBusElapsed >= 150 || lastLCDUpdate == 0) {
    char str[50];
    snprintf(str, 50, "LCD: %s", status.lcd);
    sendMessage(str);
  }
}

void AccentaG4::sendCommand(char key) {
  serial.stopListening();                   // half-duplex bus
  serial.write9(K_COMMAND | 0x100);         // K cmd: emulate MARK parity
  serial.write9(key & 0xff);                // key: emulate SPACE parity
  serial.write9((K_COMMAND + key) & 0xff);  // checksum: emulate SPACE parity
  serial.listen();
}

void AccentaG4::sendCommands() {
  // send keypad commands
  unsigned long now = millis();
  unsigned long elapsed = now - tx.start;

  if (tx.queue.count() && elapsed >= K_DELAY_MS) {  // throttle tx
    sendCommand(tx.queue.pop());
    tx.start = now;
  }
}

void AccentaG4::queryStatus() {
  sendPanelSignals();
  sendLedStatus();
  sendLcdStatus();
}

boolean AccentaG4::validateChecksum(char expectedChecksum) {
  char checksum = 0;
  for (int i = 0; i < rx.ptr; i++) {
    checksum += rx.data[i];
  }
  return checksum == expectedChecksum;
}



void AccentaG4::loop() {
  readPanelSignals();

  long lastReadBusElapsed = millis() - lastReadBus;

  if (lastReadBusElapsed >= 100 || lastReadBus == 0) {
    readBusMessages();
    sendCommands();
  }
}

#include "AccentaG4.h"
#include "SerialBridge.h"

#define COMMS_RX_PIN 10
#define COMMS_TX_PIN 11
#define SIG_ABORT_PIN 2
#define SIG_SET_PIN 3
#define SIG_INT_PIN 4
#define SIG_PA_PIN 5
#define LINK_SPEED 115200
#define CONSOLE_SPEED 115200
#define SENSORS_INTERVAL_MS 5000
#define ENABLE_CHAR '+'
#define DISABLE_CHAR '-'
#define ENABLE_GRACE_MS 1500

void enableHandler(HardwareSerial serial, char enableChar, boolean justEnabled);
void disableHandler(HardwareSerial serial, char disableChar, boolean justDisabled);
void readHandler(HardwareSerial serial, char c);
void sendMessage(String msg);

SerialBridge bridge(Serial, LINK_SPEED, LED_BUILTIN, ENABLE_CHAR, DISABLE_CHAR,
                    ENABLE_GRACE_MS, enableHandler, disableHandler, readHandler);

AccentaG4 accentaG4(COMMS_RX_PIN, COMMS_TX_PIN, SIG_SET_PIN, SIG_ABORT_PIN,
                    SIG_INT_PIN, SIG_PA_PIN, sendMessage);



void enableHandler(HardwareSerial serial, char enableChar, boolean justEnabled) {
  serial.println(enableChar);
}

void disableHandler(HardwareSerial serial, char disableChar, boolean justDisabled) {
  serial.println(disableChar);
}

void readHandler(HardwareSerial serial, char c) {
  if (c == '?') {
    accentaG4.queryStatus();
  } else {
    accentaG4.sendKey(c);
  }
}

void sendMessage(String msg) {
  if (bridge.isEnabled()) {
    bridge.serial.println(msg);
    bridge.serial.flush();
  }
}

void setup() {
  bridge.begin();
  accentaG4.begin();
}

void loop() {
  bridge.loop();
  accentaG4.loop();
}

/*
  SerialBridge.cpp - mcu <=> mpu serial bridge
  Created by Luca Paolini, April 3, 2019.
*/

#include "SerialBridge.h"

SerialBridge::SerialBridge(
  HardwareSerial &serial, long linkSpeed, int statusLed,
  char enableChar, char disableChar, unsigned long enableGraceMillis,
  void (*enableHandler)(HardwareSerial serial, char enableChar, boolean justEnabled),
  void (*disableHandler)(HardwareSerial serial, char disableChar, boolean justEnabled),
  void (*readHandler)(HardwareSerial serial, char readChar))
  : serial(serial) {
  this->linkSpeed = linkSpeed;
  this->statusLed = statusLed;
  this->enableChar = enableChar;
  this->disableChar = disableChar;
  this->enableGraceMillis = enableGraceMillis;
  this->enableHandler = enableHandler;
  this->disableHandler = disableHandler;
  this->readHandler = readHandler;
}

void SerialBridge::start() {
  serial.begin(linkSpeed);
}

void SerialBridge::stop() {
  serial.end();
}

boolean SerialBridge::isEnabled() {
  boolean withinGracePeriod = (enableGraceMillis == 0) || (millis() - lastEnabled < enableGraceMillis);
  return enabled && withinGracePeriod;
}

void SerialBridge::read() {
  int c = serial.read();
  if (c == enableChar) {
    boolean wasEnabled = enabled;
    enabled = true;
    lastEnabled = millis();
    blink();
    if (enableHandler) {
      enableHandler(serial, enableChar, wasEnabled == false);
    }
    return;
  }
  if (c == disableChar) {
    boolean wasEnabled = enabled;
    enabled = false;
    if (disableHandler) {
      disableHandler(serial, disableChar, wasEnabled == true);
    }
    return;
  }
  if (isEnabled()) {
    if (readHandler) {
      readHandler(serial, c);
    }
  };
}

void SerialBridge::blink() {
  lastBlink = millis();
  ledState = HIGH;
  digitalWrite(statusLed, ledState);
}

void SerialBridge::resetBlink() {
  unsigned long now = millis();
  unsigned long duration = now - lastBlink;
  if (ledState == HIGH && duration > BLINK_DURATION_MS) {
    ledState = LOW;
    digitalWrite(statusLed, ledState);
    lastBlink = now;
  }
}

void SerialBridge::begin() {
  pinMode(statusLed, OUTPUT);
  digitalWrite(statusLed, HIGH);
  delay(2500);
  start();
  digitalWrite(statusLed, LOW);
}

void SerialBridge::end() {
  stop();
  digitalWrite(statusLed, LOW);
}

void SerialBridge::loop() {
  read();
  resetBlink();
}

[J-M-L]

may be not your issue but you should not handle pins in the constructor as it's called before main() is call, meaning before the Arduino code can setup the environment.

@ageis

  pinMode(setPin, INPUT_PULLUP);
  pinMode(abortPin, INPUT_PULLUP);
  pinMode(intPin, INPUT_PULLUP);
  pinMode(paPin, INPUT_PULLUP);

if you are unlucky and have a global AccentaG4 instance, the pullup could be removed by code running later that is setting up the board.

➜ typical approach is to add a begin() method which you call in the setup() and this is where you'll do the INPUT_PULLUP thingy.

so

AccentaG4::AccentaG4(uint8_t rxPin, uint8_t txPin, uint8_t setPin,
                     uint8_t abortPin, uint8_t intPin, uint8_t paPin,
                     void (*sendMessage)(String msg))
  : serial(rxPin, txPin) {
  this->setPin = setPin;
  this->abortPin = abortPin;
  this->intPin = intPin;
  this->paPin = paPin;
  this->sendMessage = sendMessage;
}

void AccentaG4::begin() {
  pinMode(setPin, INPUT_PULLUP);
  pinMode(abortPin, INPUT_PULLUP);
  pinMode(intPin, INPUT_PULLUP);
  pinMode(paPin, INPUT_PULLUP);

  serial.begin(BUS_SPEED);
  tx.start = millis();
  queryStatus();
}

(and I'm not sure about the : serial(serial) )

[ageis]

quick question is the println function asyncronous ? if so is there a way I can force it to send immediately and not do anything else till it's sent.

[J-M-L]

it is. if you want to wait until everything got sent, call flush()

Serial.println("Hello world"); // add "Hello world\r\n" to the outgoing buffer
Serial.flush(); // wait until the buffer is empty

[ageis]

flush appears to make the situation worse. I am thoroughly confused

[J-M-L]

blocking the code is usually a bad idea

As discussed before, the constructors are called before main

@ageis

SerialBridge bridge(Serial, LINK_SPEED, LED_BUILTIN, ENABLE_CHAR, DISABLE_CHAR,
                    ENABLE_GRACE_MS, enableHandler, disableHandler, readHandler);

so when you do this Serial might not be setup yet.

[ageis]

the serial is initialised by the start method in SerialBridge class. It doesn't seem to function without inheriting from the serial.

[J-M-L]

is there a GitHub for the whole project you got?

have you tried suggestion from post 4?

[ageis]

@ageis

SerialBridge bridge(Serial, LINK_SPEED, LED_BUILTIN, ENABLE_CHAR, DISABLE_CHAR,
                    ENABLE_GRACE_MS, enableHandler, disableHandler, readHandler);

GitHub - lpaolini/AccentaG4: Virtual keypad for Honeywell Security Accenta G4...

I have moved the pin setup to a begin function.

[J-M-L]

do you have the exact same hardware? they use 9 bit serial emulation.

they also mention in the GitHub

WARNING: the original version of SoftwareSerial9 by addible had a bug in the recv() method, fixed by edreanernst in his fork, then fixed by addible too. Both work fine, just make sure to use the latest version.

they also describe a circuit for the enhanced interface using opto-isolation

➜ do you have that?

[ageis]

Yes I use his optocoupler design.

I can send and receive messages from the alarm panel. I can also receive the hardware signals such as abort, intruder.

The first two optocoupler are for sending and receiving messages. The rest are outputs on the panel to signal various conditions such as intruder.

Hmm how do I check am using the latest version of the software serial library?

[J-M-L]

@ageis

Hmm how do I check am using the latest version of the software serial library?

go to GitHub - addibble/SoftwareSerial9: 9-bit serial output hack for Arduino and download the zip and add it manually to your list of know libraries

[ageis]

i have changed the software9serial library, it still malfunctioning.

[J-M-L]

OK unfortunately it's hard to debug without the config.

try to add a small delay at the start of the setup in case "things need to settle"