Mega2650 -> Xbee -> Computer serial hiccups at higher throughput

Hello all,

The arduino is collecting data remotely and transmitting back to the computer.

The payload is around 100 bytes, so I increased the send buffer to 128 so the print would not block.

all baud rates are 9600

right now the send is timed to once a second, and it works fine.

As I reduce the time between sends, it starts to choke and data to the computer becomes erratic. dropped characters etc etc. Even when the arduino is switched off, the receive side still outputs data for a few seconds. This would indicate it must be buffered somewhere between the receiving xbee and windows application correct?

I am using a cheap amazon xbee usb adaptor on the computer side, which I presume has its own buffer before it sends over USB? (not accurate statement)

I have tried messing with the settings in windows device manager with no change.

Here is some code stripped down. If you imagine other methods are updating these variables in background, the attached is simulating the sending over serial to the xbee link.

Adafruit_GPS GPS(&Serial2);

uint32_t message_timer = millis();
struct Point q2 = {0, 0};
int lap = 0;
String lap_string;
String lap_change_string;
float ax_conv, ay_conv;
float GPS_speed = 0;
int obd_rpm;
int obd_coolant_temp;
float obd_battery;


void setup() {
  // put your setup code here, to run once:
  Serial3.begin(9600);
}

void loop() {
  // put your main code here, to run repeatedly:
  if (millis() - message_timer > 1000) {
    Send_message();
    message_timer = millis();
  }
}

void Send_message()
{
  Serial3.print("A");   Serial3.print(",");
  Serial3.print(GPS.fix); Serial3.print(",");         
  Serial3.print(q2.x, 8); Serial3.print(",");         
  Serial3.print(q2.y, 8); Serial3.print(",");      
  Serial3.print(lap); Serial3.print(",");
  Serial3.print(ax_conv); Serial3.print(",");
  Serial3.print(ay_conv); Serial3.print(",");
  Serial3.print(lap_string); Serial3.print(",");
  Serial3.print(lap_change_string); Serial3.print(",");
  Serial3.print(GPS_speed, 0); Serial3.print(",");
  Serial3.print(obd_rpm); Serial3.print(",");
  Serial3.print(obd_coolant_temp); Serial3.print(",");
  Serial3.print(obd_battery); Serial3.print(",");
  Serial3.print("Z");

}

resulting serial data at the computer

1 second delay between transmissions - normal

timestamp          payload
05.54.40.165    A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.54.41.169 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.54.42.174 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.54.43.178 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z

300 msec between transmissions

timestamp          payload
05.59.19.754 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.20.726 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.20.796 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.20.888 A,0,0.00000000,0.0000A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.21.958 ERROR0
05.59.22.029 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.22.159 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.23.098 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.23.166 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.23.296 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.24.366 ERROR0
05.59.24.541 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.24.670 A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,A,0,0.00000000,0.00000000,0,0,0,----------,----------,0,0,0,0.00,Z
05.59.25.741 ERROR0

note: i have no idea where the ERROR0 is coming from it is not my code...

I would like to change the timing to be as frequent as possible, using Serial.availableForWrite() or similar and only pushing into the Tx buffer when there is room and it won't block. but I am stumbling here.

Please let me know anything that comes to mind.

in the meantime, i will try using an uno as the usb adaptor instead of the amazon xbee adaptor

Wild guess: the code running on your PC that accepts the data isn't fast enough to handle such a throughput and you are getting an input buffer overflow on your PC. Again, just a wild guess.

Power_Broker:
Wild guess: the code running on your PC that accepts the data isn't fast enough to handle such a throughput and you are getting an input buffer overflow on your PC. Again, just a wild guess.

I have tried using normal terminal emulator/serial monitor as well as my application and the response is the same. I don't think this is an issue because 9600 baud should be snail pace to any computer

I am using a cheap amazon xbee usb adaptor on the computer side, which I presume has its own buffer before it sends over USB?

Why would you assume that? It almost certainly does NOT.

I do hope that the fine folks at http://snippets-r-us.com can help you with your snippet(s).

PaulS:
Why would you assume that? It almost certainly does NOT.

I do hope that the fine folks at http://snippets-r-us.com can help you with your snippet(s).

Good to know about the USB adapter. For sake of troubleshooting, I will remove it anyway to rule it out as an issue.

Also I will update my code "snippet" per your suggestion.

Thanks

I have never used an XBee (too expensive). Surely you can use a much higher baud rate?

With the regular USB cable my normal communication with my PC is at 500,000 baud.

...R

Robin2:
I have never used an XBee (too expensive). Surely you can use a much higher baud rate?

With the regular USB cable my normal communication with my PC is at 500,000 baud.

...R

The radios are at a low rate to help increase range. I believe the PC-USB baud rate needs to match the receiving xbee, but I will try that to be sure.

I imagine that the rate of wireless transmission is quite separate from the baud rate.

Of course it would not make sense to send more data per second than the wireless side can transmit. Presumably the XBee has a buffer in which to store a certain amount of data received via the serial connection.

For example an Uno has a 64 byte serial output buffer and it can put (say) 50 bytes into that buffer at high speed (assuming it is empty) even if the baud rate is low. But if it were to try to put a second 50 bytes into the buffer it would have to wait until enough bytes had been sent onwards by the USART - which does depend on the baud rate.

On the receiving side I would expect a higher baud rate to be better as it would take the received data away as quickly as possible.

It is essential when receiving serial data to understand that it probably won't all arrive at once because an Arduino and especially a PC can collect data much faster than it can arrive even at a high baud rate.

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. The concepts can also be used in PC programs.

...R

Robin2:
I imagine that the rate of wireless transmission is quite separate from the baud rate.

Of course it would not make sense to send more data per second than the wireless side can transmit. Presumably the XBee has a buffer in which to store a certain amount of data received via the serial connection.

For example an Uno has a 64 byte serial output buffer and it can put (say) 50 bytes into that buffer at high speed (assuming it is empty) even if the baud rate is low. But if it were to try to put a second 50 bytes into the buffer it would have to wait until enough bytes had been sent onwards by the USART - which does depend on the baud rate.

On the receiving side I would expect a higher baud rate to be better as it would take the received data away as quickly as possible.

It is essential when receiving serial data to understand that it probably won't all arrive at once because an Arduino and especially a PC can collect data much faster than it can arrive even at a high baud rate.

Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. The concepts can also be used in PC programs.

...R

Yes on transmit side, I think I am OK. The transmit buffer is large enough for the whole payload, and I don't write again until it is surely empty.

Seeing that when the transmitter is turned off, data still arrives at the PC it points to receive side obviously.

I will try faster USB baud rate. I don't know how the xbee usb adapter negotiates baud rate with the xbee, but maybe there is some handshaking I don't know of.

Brief update:

on the receiving side: I removed the Xbee usb adapter, and replaced it with an Uno acting as the usb adapter.

xbee - 9600 - uno sw serial / uno hw serial - 38400 to the computer

possible slightly better results, but still largely the same response. Garbage when transmit interval < 400 msec. So I don't think the usb adapter was the issue.

I falsely assumed if everything is 9600 and I make sure I don't overflow the tx buffer on the transmitting arduino, everything should make it out the other side at the same rate. It seems the RF data rate of 9600 baud does not include some overhead, so when I try to send at this full speed it chokes somewhere.

I will experiment with packet sizes on the RF to make this more reliable but my conclusion at the moment is I need to keep the transmit interval a little slower. I cannot treat the RF link like a UART and send 9600 baud through it and expect it to be the same on the other end.

thanks all for your help. If I get any posotive results from configuring the XBee's i will update.

Hopefully someone who does know about XBees will come to the rescue.

...R

I don't know how the xbee usb adapter negotiates baud rate with the xbee

It doesn't. The adapter is just a pass-through device, connecting the com port on the PC to the XBee. It is the responsibility of the application on the PC to select the proper baud rate to talk to the XBee.

You haven't said anything about the type of XBees you are using (Series 1/point to point or Series 2/mesh network) or about how they are configured (other than baud rate).

I do hope you didn't drink the digi KoolAid. They claim that the XBees can be used out of the box with no configuration. While that can happen, the amount of data that can be sent back and forth is way less than the amount that can be sent if the XBees are properly configured (to talk to ONE specific other device).

PaulS:
It doesn't. The adapter is just a pass-through device, connecting the com port on the PC to the XBee. It is the responsibility of the application on the PC to select the proper baud rate to talk to the XBee.

Makes sense

PaulS:
You haven't said anything about the type of XBees you are using (Series 1/point to point or Series 2/mesh network) or about how they are configured (other than baud rate).

I do hope you didn't drink the digi KoolAid. They claim that the XBees can be used out of the box with no configuration. While that can happen, the amount of data that can be sent back and forth is way less than the amount that can be sent if the XBees are properly configured (to talk to ONE specific other device).

I may have sipped. It did work easily, but as you say there should be room for improvement. I will examine this configuration.

Thanks for the insights

Update to this:

PaulS was exactly right, the Xbee was set in broadcast mode (no destination address). One line in the docs I must have missed is that it sends broadcast messages 4 times, so my throughput was cut in 4.

Set the destination address correctly and now it can transmit the payload ~10 times a second which is great.

Thanks to all who contributed.