Huge Latency + Errors when Sending Analog Values using XBees

Projects Networking, Protocols, and Devices
1

Dear all,

I am building a wireless musical interface using three XBees and two Arduinos. When I try to send analog potentiometer values using the Arduino XBee Library, I run into huge (+10 second) latency and errors. Why is this happening? Has anyone encountered this problem before? My wireless hardware setup involves:

  • One XBee 3 module set up as a coordinator running in API 1 Mode (without escapes). This module is connected to a Sparkfun USB explorer which is connected to my Macbook.
  • Two XBee 3 modules set up as routers running in transparent mode.
  • Two Arduinos connected to the XBee routers

All XBee 3 modules are configured to run using 802.15.4 firmware. Everything works fine when I am simply running two modules in transparent mode and just printing serial values. Everything also works fine when I am running the two routers in transparent mode and one coordinator in API mode and send unchanging integer values using the Arduino XBee library. The latency/error issue arises when I am reading and sending a variable potentiometer value using the Arduino XBee library. Numbers received on my Macbook are totally mangled and often take 5-10 seconds to update and correspond to the physical potentiometer position. I've pinpointed the issue to the code running on the Arduinos. It seems to me as if the XBee library somehow can't keep up with the rate of transmission. Has anyone ever encountered this problem before? I'm going to wait a couple of days and then attempt to bypass the library and write my own packet sending functions on the Arduino to see if this solves the issue. Any advice is appreciated. My Arduino code is attached below for reference.

Thanks in advance,
Bernard Le Poisson

#include <XBee.h>

XBee xbee = XBee();
uint8_t payload[] = {0, 0};
int pot;

// 16-bit addressing: Enter address of remote XBee, typically the coordinator
Tx16Request tx = Tx16Request(0000, payload, sizeof(payload));

void setup()
{
  //Startup delay
  delay(5000);
  //Begin HW serial
  Serial1.begin(115200);
  xbee.setSerial(Serial1);
}

void loop() {

    //Break down 10-bit analog reading into two bytes and place in payload
    pot = analogRead(A0);
    payload[0] = pot >> 8 & 0xff;
    payload[1] = pot & 0xff;
    //Send to coordinator
    xbee.send(tx);
    delay(10);
}
2

Did you mean to say:

   payload[1] = pot >> 8 & 0xff;
   payload[0] = pot & 0xff;

instead? Are you sure of the correct byte order?

3

bernard_le_poisson:
I'm going to wait a couple of days and then attempt to bypass the library and write my own packet sending functions on the Arduino to see if this solves the issue.

If you bypass the XBee lib, you don't have to write your own code. You can use SerialTransfer.h to automatically packetize and parse your data for inter-Arduino communication over XBees without the headace. The library is installable through the Arduino IDE and includes many examples.

Here are the library's features:

This library:

  • can be downloaded via the Arduino IDE's Libraries Manager (search "SerialTransfer.h")
  • works with "software-serial" libraries
  • is non blocking
  • uses packet delimiters
  • uses consistent overhead byte stuffing
  • uses CRC-8 (Polynomial 0x9B with lookup table)
  • allows the use of dynamically sized packets (packets can have payload lengths anywhere from 1 to 255 bytes)
  • can transfer bytes, ints, floats, and even structs!!

Example TX Arduino Sketch:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  myTransfer.txBuff[0] = 'h';
  myTransfer.txBuff[1] = 'i';
  myTransfer.txBuff[2] = '\n';
  
  myTransfer.sendData(3);
  delay(100);
}

Example RX Arduino Sketch:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

void setup()
{
  Serial.begin(115200);
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop()
{
  if(myTransfer.available())
  {
    Serial.println("New Data");
    for(byte i = 0; i < myTransfer.bytesRead; i++)
      Serial.write(myTransfer.rxBuff[i]);
    Serial.println();
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
}
4

Power_Broker:
If you bypass the XBee lib, you don't have to write your own code. You can use SerialTransfer.h to automatically packetize and parse your data for inter-Arduino communication over XBees without the headace. The library is installable through the Arduino IDE and includes many examples.

Thanks Power_Broker! I switched to this library, which gave me slightly shorter packets (11 vs 16). What seemed to make a really big difference, however, was increasing my delay value to 100ms. The values are coming in fine now and I can get two potentiometer values from separate XBees. A 100ms delay is not ideal, especially not for live music, so this will likely inform the design of my interface. Going any lower e.g 10-50ms results in errors, so there does appear to be a limit on the packet throughput of the XBees.

5

bernard_le_poisson:
Thanks Power_Broker! I switched to this library, which gave me slightly shorter packets (11 vs 16). What seemed to make a really big difference, however, was increasing my delay value to 100ms.

I've been able to transmit bigger packets with ~10ms delay or less with SerialTransfer.h. Most likely the delay/errors are caused by your code. Can you post your updated code?

6

Power_Broker:
I've been able to transmit bigger packets with ~10ms delay or less with SerialTransfer.h. Most likely the delay/errors are caused by your code. Can you post your updated code?

Oh, interesting! I'm all ears. To be clear, I'm receiving the packets not via an Arduino, but via a Sparkfun USB Explorer talking to the Python XBee library. Happy to provide that code if you want as well. Here's my updated Arduino code:

#include <SerialTransfer.h>

SerialTransfer myTransfer;
uint8_t pot = 0;

void setup()
{
  //Startup delay
  delay(5000);
  //Begin HW serial
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop() {
    pot = analogRead(A0)*128/1024;
    myTransfer.txBuff[0] = pot;
    myTransfer.sendData(1);
    delay(100);
}
7

bernard_le_poisson:
Oh, interesting! I'm all ears. To be clear, I'm receiving the packets not via an Arduino, but via a Sparkfun USB Explorer talking to the Python XBee library.

Yeah, that's why earlier I said you need to use the pySerialTransfer Python package instead. Please post all of your code on both sides so I can see where the problem is.

8

Power_Broker:
Yeah, that's why earlier I said you need to use the pySerialTransfer Python package instead. Please post all of your code on both sides so I can see where the problem is.

So here's my code from the pySerialTransfer Python package:

from time import sleep
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer("/dev/cu.usbserial-D3071XI1")
        link.open()
        sleep(2) # allow some time for the Arduino to completely reset
        
        while not link.available():
            if link.status < 0:
                print('ERROR: {}'.format(link.status))
            
        print('Response received:')
        
        response = ''
        for index in range(link.bytesRead):
            response += chr(link.rxBuff[index])
        print(response)
        
    except KeyboardInterrupt:
        link.close()

and from Arduino:

#include <SerialTransfer.h>

SerialTransfer myTransfer;
static const uint8_t aPins[] = {A0,A1,A2,A3,A4,A5};
uint8_t pVal [6];

void setup()
{
  //Startup delay
  delay(5000);
  //Begin HW serial
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}

void loop() {
    myTransfer.txBuff[0] = "^";
    for (int i = 0; i < 6; i++) {
      pVal[i] = analogRead(aPins[i])*128/1024;
      myTransfer.txBuff[i + 1] = pVal[i];
    }
    myTransfer.sendData(7);
    delay(100);
}

At the moment I'm getting a long list of "ERROR: -1"s, and very rarely do I get a successful response. Python is talking to the USB Explorer fine via the XBee Python library at the moment, so my serial communication is fine. thanks PowerBroker!

9

I'll take a look at it in a bit, but you should really post your code wrapped with the proper tags instead of however you've been posting it.

10

The python you posted only reads a packet from the Arduino once - how are you testing the speed at which you receive multiple packets?

11

Power_Broker:
The python you posted only reads a packet from the Arduino once - how are you testing the speed at which you receive multiple packets?

I'm unsure how to receive multiple packets with pySerialTransfer. How would you write that? But when using the Python XBee library, I'm testing by printing every received packet to the console (this creates a steady stream of incoming packets). Then I turn my potentiometers and see at which point the packets become garbled i.e packet values do not correspond to the current potentiometer position.

12

bernard_le_poisson:
I'm unsure how to receive multiple packets with pySerialTransfer. How would you write that?

'link.available()' parses and receives new packets. Call it whenever you want to search for a new packet.

Try this code:

Arduino:

#include <SerialTransfer.h>


const uint8_t arrSize = 6;


SerialTransfer myTransfer;
static const uint8_t aPins[arrSize] = {A0, A1, A2, A3, A4, A5};
uint16_t pVal[arrSize];


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}


void loop()
{
  for (int i = 0; i < arrSize; i++)
    pVal[i] = analogRead(aPins[i]);

  myTransfer.txObj(pVal, sizeof(pVal));
  myTransfer.sendData(sizeof(pVal));
}

Python:

from time import sleep
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM17')
        link.open()
        sleep(2) # allow some time for the Arduino to completely reset
        
        while True:
            while not link.available():
                if link.status < 0:
                    print('ERROR: {}'.format(link.status))
               
            print('Response received:')
           
            an_port = 0
            for index in range(0, link.bytesRead, 2):
                print('A{}: {}'.format(an_port, (link.rxBuff[index+1] << 8) | link.rxBuff[index]))
                an_port += 1
            print('')
       
    except KeyboardInterrupt:
        link.close()
13

Power_Broker:
Try this code:

Thanks for all the help. I really appreciate it. I love the way you're formatting the packets and reading them in, that's helpful to see. Unfortunately, I keep getting "ERROR: -1" messages most of the time when running this code. Every now and then a packet comes through successfully with the proper potentiometer values. But most of the time the link isn't available for some reason. Interestingly enough, a successful packet is preceded by a single "ERROR: -3" message. What is the difference between a link.status of -1 and -3? Why am I mostly not receiving responses? I tried adding and changing delay values but that didn't make a difference regarding how many successful responses I received...

EDIT: I took a look at SerialTransfer.h. Apparently I'm getting plenty of CRC errors (-1), and then immediately before a successful packet, a Stop Byte Error (-3). This reminds me that I was getting a lot of checksum errors when my coordinator XBee was in API 2 mode (with escaped characters). When I switched my coordinator to API 1 mode (without escaped characters) the checksum errors stopped. I'm going to switch the coordinator back to API 2 to see what happens.

EDIT 2: API 2 didn't seem to make any difference. Is there any way to ignore the CRC error?

14

I tested the code in my last reply with a Mega and Python 3.7 with no errors at all. Using it with XBees in between shouldn't change anything. Since you're still getting errors regardless, I'll do some testing with my own XBees to make sure (if I can find my XBee USB dongle, lol).

bernard_le_poisson:
EDIT: I took a look at SerialTransfer.h. Apparently I'm getting plenty of CRC errors (-1), and then immediately before a successful packet, a Stop Byte Error (-3). This reminds me that I was getting a lot of checksum errors when my coordinator XBee was in API 2 mode (with escaped characters). When I switched my coordinator to API 1 mode (without escaped characters) the checksum errors stopped. I'm going to switch the coordinator back to API 2 to see what happens.

Wait a second... Your XBees are in API mode? They should be in transparent mode if you want to use these libraries.

15

Power_Broker:
Wait a second... Your XBees are in API mode? They should be in transparent mode if you want to use these libraries.

My coordinator is in API 1 mode and my two routers are in transparent mode, specifically because I want to receive different sensor values from each of them and be able to differentiate who is sending what (via their 16bit address). I'm guessing this isn't possible with this library then?

16

That information was not captured in the code in your original post, so I missed it. When asking questions on the forum, we need as much detailed and accurate info as possible about your project so we can help you quickly and effectively.

That being said, can you post a diagram of your connections (including PC XBee) so I can get an accurate idea of the flow of data? Also, can you post the current version of all of your sketches/scripts you're using?

17

Power_Broker:
That information was not captured in the code in your original post, so I missed it. When asking questions on the forum, we need as much detailed and accurate info as possible about your project so we can help you quickly and effectively.

That being said, can you post a diagram of your connections (including PC XBee) so I can get an accurate idea of the flow of data? Also, can you post the current version of all of your sketches/scripts you're using?

Sorry about that. I should have been more explicit about what modes my XBees were in and what I was trying to achieve. Here's a diagram of my connections. Data flows from the bottom of the diagram to the top. Do you think it's possible to achieve my goal at low latency using either the SerialTransfer library or the Python XBee library? Or, does differentiating between the routers' 16bit addresses with the coordinator in API mode inherently add latency?

1584×1224 496 KB

Arduino Sketch:

#include <SerialTransfer.h>

const uint8_t arrSize = 6;

SerialTransfer myTransfer;
static const uint8_t aPins[arrSize] = {A0, A1, A2, A3, A4, A5};
uint16_t pVal[arrSize];

void setup()
{
  // 5 second startup delay
  delay(5000);
  // open HW serial
  Serial1.begin(115200);
  myTransfer.begin(Serial1);
}


void loop()
{
  for (int i = 0; i < arrSize; i++) {
    pVal[i] = analogRead(aPins[i]);
  }
  myTransfer.txObj(pVal, sizeof(pVal));
  myTransfer.sendData(sizeof(pVal));
  // 10 ms delay
  delay(10);
}

and Python SerialTransfer script:

from time import sleep
from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer("/dev/cu.usbserial-D3071XI1")
        link.open()
        sleep(2) # allow some time for the Arduino to completely reset
       
        while True:
            while not link.available():
                if link.status < 0:
                    print('ERROR: {}'.format(link.status))
               
            print('Response received:')
           
            an_port = 0
            for index in range(0, link.bytesRead, 2):
                print('A{}: {}'.format(an_port, (link.rxBuff[index+1] << 8) | link.rxBuff[index]))
                an_port += 1
            print('')
       
    except KeyboardInterrupt:
        link.close()

Here's my script that fully works (but only at 100ms latency) using the Python XBee library and the SerialTransfer.h Arduino Library (which appears to be working, even with the coordinator in API 1 Mode):

import serial
from digi.xbee.devices import XBeeDevice
device = XBeeDevice("/dev/cu.usbserial-D3071XI1", 115200)
device.open()

while True:
    xbee_message = device.read_data()
    if xbee_message is not None:
        if str(xbee_message.remote_device.get_16bit_addr()) == "15 49":
            print(list(xbee_message.data)[3:9])
        elif str(xbee_message.remote_device.get_16bit_addr()) == "16 38":
           print(list(xbee_message.data)[3:9])
18

Ok, thanks for clearing things up.

I'm not sure how the coordinator can deconflict packet collisions if the routers are in transparent mode (i.e. not using standard XBee packets).

You'll probably be better off using the standard XBee libraries for this point-to-mulipoint mesh. It is possible, however, to use SerialTransfer, but you'll have to time the transmission/reception of each router packet similarly to how networked LoRa radios do packet timing. You can also send the address of each device using SerialTransfer by sending structs designed like this:

struct STRUCT {
  uint16_t addr;
  uint16_t an_0;
  uint16_t an_1;
  uint16_t an_2;
  uint16_t an_3;
  uint16_t an_4;
  uint16_t an_5;
} testStruct;