Xbee Data getting cut-off

Hello,

I am trying to send data from a Mega 2560 via an Xbee Shield however, the data is being cut off. The data should look like:

COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:27,
COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:38,
COOP-04191795B66,N/A,Food_1,N/A,5/27/2016,20:41:40,
COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:42,
COOP-04191795B66,N/A,Food_1,N/A,5/27/2016,20:41:43,
COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:45,

And for the first few hours it does. However, after that the data starts looking like:

COOP-0419179FDEE,N/A,Food_2,N/A,COOP-04191798B11,N/A,Food_2,N/A,COOP-04191798B11,N/A,Food_2,N/A,5/28/2016,5:32:4,
5/28/2016,5:32:9,
5/28/2016,5:32:22,
COOP-04191798B11,N/A,Food_2,N/A,5/28/2016,5:32:43,

I didn’t post the entire code below, this is just a sample of one Serial Port:

void readRfidPort3 () { // 3rd Pin (RX = pin 17 & TX = pin 16)
  if(Serial2.available() > 0) { // if data available from reader
    if((val = Serial2.read()) == 2) { // check for header
    bytesread = 0;
    while(bytesread<10) { // read 10 digit code
      if( Serial2.available() > 0) {
        val = Serial2.read();
          if((val == 2)||(val == 3)) { // if header or stop bytes before the 10 digit reading
          break; // stop reading
          }
          code[bytesread] = val; // add the digit
          bytesread++; // ready to read next digit
      }
    }
    if(bytesread == 10) { // if 10 digit read is complete
    //codeToSd();
    //timeToSd();
     Serial.print("RFID Aviary-Tag: ");
    Serial.print(Avy);
    Serial.print("-");
    Serial.println(code);
    Serial.print("No Data: ");
    Serial.println(BkN);
    Serial.print("Location & Location Type: "); 
    Serial.print(P2_L);
    Serial.print(" | ");
    Serial.println(P2_T);
    Serial.print("Writing Time to File: ");  
    digitalWrite(13, HIGH); //indicate it by glowing LED at pin 13
        delay(1500); // delay 1.5 seconds
        dataFile.print(Avy);
        dataFile.print("-");
        dataFile.print(code);
        dataFile.print(",");
        dataFile.print(BkN);
        dataFile.print(",");
        dataFile.print(P2_L);
        dataFile.print(",");
        dataFile.print(P2_T);
        dataFile.print(",");
            XBee.print(Avy);
            XBee.print("-");
            XBee.print(code);
            XBee.print(",");
            XBee.print(BkN);
            XBee.print(",");
            XBee.print(P2_L);
            XBee.print(",");
            XBee.print(P2_T);
            XBee.print(",");          
    timeToSd();
    dataFile.flush();
    digitalWrite(13, LOW); // glow off LED
    }
    bytesread = 0;
    }
  
}
}

The TimetoSD is pretty straight forward, mostly a copy & paste from the ADAfruit website I believe:

void timeToSd() {
      if (dataFile) {
        //Serial.begin(57600);
        DateTime now = RTC.now();
        Serial.print(now.month(), DEC);
        Serial.print("/");
        Serial.print(now.day(), DEC);
        Serial.print("/");
        Serial.print(now.year(), DEC);
        Serial.print(" | ");
        Serial.print(now.hour(), DEC);
        Serial.print(":");
        Serial.print(now.minute(), DEC);
        Serial.print(":");
        Serial.println(now.second(), DEC);
        Serial.println("");
          dataFile.print(now.month(), DEC);
          dataFile.print('/');
          dataFile.print(now.day(), DEC);
          dataFile.print('/');
          dataFile.print(now.year(), DEC);
          dataFile.print(',');
          dataFile.print(now.hour(), DEC);
          dataFile.print(':');
          dataFile.print(now.minute(), DEC);
          dataFile.print(':');
          dataFile.print(now.second(), DEC);
          dataFile.println(","); //"println" should be at the end, not at the "Time: "
                XBee.print(now.month(), DEC);
                XBee.print('/');
                XBee.print(now.day(), DEC);
                XBee.print('/');
                XBee.print(now.year(), DEC);
                XBee.print(',');
                XBee.print(now.hour(), DEC);
                XBee.print(':');
                XBee.print(now.minute(), DEC);
                XBee.print(':');
                XBee.print(now.second(), DEC);
                XBee.println(",");  
        } else {
        // if the file didn't open, print an error:
        Serial.println("error opening test.txt");
        }
}

One concern I had is the Xbee shield cannot handle three serial connections, as this is too much data whereas the SD data logger can keep up (I forgot to mention the SD Datalogger works perfectly fine). Anyway, I am looking forward to any suggestions the community may have.

Thank you.

One concern I had is the Xbee shield cannot handle three serial connections

It can only handle one.

You might get some help at http://snippets-r-us.com

Well that is strange because I am collecting data from 3 antennas without any issue for the most part. The only issue, as described above is every once in a while the data gets really messed up. What's strange is the volume has nothing to do with this. Sometimes there is really high volume and everything works great whereas, low volume has issue (and vice versa).

I am trying to think about what issues might be causing this however, I am at a loss.

I get the impression that you are not checking to see if you have received a complete message. How is the start and end of the message identified?

If you don't check that you have received a complete message you risk getting them all mixed up.

Have a look at the examples in Serial Input Basics. From reading other Threads I believe they are also relevant to XBees

...R

Well that is strange because I am collecting data from 3 antennas without any issue for the most part.

How? I would guess that you are collecting data from ONE XBee that is getting data from more than one other XBee.

But, I don't care for guessing games. Describe you complete project and post ALL of your code.

Thanks for the advice. I will look into making sure it checks all the data is received. However, I am not sure how I do that.

I will try to describe the entire project here. At a high level I am trying to collect movement patterns of our subjects using RFID antennas.

I have a homemade RFID antenna collecting data from RFID pit tags attached to the subjects. The data is read using an (RFID Module. The RFID module connects to the serial ports on an Mega 2560. Each Mega has a total of four antennas & microchips connected to the four serial ports. On top of the Mega is an Xbee Shield with an Xbee Wire Antenna. On top of that I have an Adafruit Data Logger that writes everything to a SD card (as a backup).

Here is a link to what it looks like (w/o the antenna as those are in the field): Prototype

One of the issues I am having, as described above is data is not coming through consistently over Xbee. The Data Logger is flawless and seems to collect everything correctly.

Had to attach the code. It was too long for the forum rules (maybe this is part of my problem).

RFID Code.txt (15.2 KB)

CodenameAter: Thanks for the advice. I will look into making sure it checks all the data is received. However, I am not sure how I do that.

You say that the Mega receives data on its 4 HardwareSerial ports. Show us an example of a single message, including whatever start- and end-markers are used.

It may be wise to build and test the system with just 3 devices so that Serial is free for debug messages.

I am not familiar with Xbees. How does the Mega send/receive data to/from the Xbee shield?

...R

SoftwareSerial XBee(2, 3);

On a Mega? NO! Read the damned documentation.

Robin2: You say that the Mega receives data on its 4 HardwareSerial ports. Show us an example of a single message, including whatever start- and end-markers are used.

It may be wise to build and test the system with just 3 devices so that Serial is free for debug messages.

I am not familiar with Xbees. How does the Mega send/receive data to/from the Xbee shield?

...R

Not sure what you mean by start- and end-markers. I provided an example of what the data should look like when a pit tag comes near the antenna.

In regards to the code, I basically used the same code as I did for the data logger but instead wrote it to XBee.

Is there something I need to code in order to set the fourth serial to debug or is that automatic when the serial is free.

The RFID modules are connected to the serial port on the Mega. The Mega is coded to write the data to the data logger and the Xbee shield. The shield sends the data to a receiving antenna connected to a PC running XCTU (or another COM port logger).

PaulS: SoftwareSerial XBee(2, 3);

On a Mega? NO! Read the damned documentation.

Can you elaborate or are you just trolling me cause I am a newbie at this?

Is there something I need to code in order to set the fourth serial to debug or is that automatic when the serial is free.

What do you mean by "the fourth serial"?

Can you elaborate or are you just trolling me cause I am a newbie at this?

https://www.arduino.cc/en/Reference/SoftwareSerial

Limitations

The library has the following known limitations:

If using multiple software serial ports, only one can receive data at a time. Not all pins on the Mega and Mega 2560 support change interrupts, so only the following can be used for RX: 10, 11, 12, 13, 14, 15, 50, 51, 52, 53, A8 (62), A9 (63), A10 (64), A11 (65), A12 (66), A13 (67), A14 (68), A15 (69).

Do you see 2 or 3 anywhere in that list?

PaulS: Do you see 2 or 3 anywhere in that list?

No, that explains why I don't receive any data from at serial. However, as of now I am actually running only three bowls so why would that impact what the other three serials ports receive and send?

In regards to a fourth serial. Robin suggested leaving one of the four serials free for debugging. I am not sure exactly what that entails. Do I simply just not use one of the four serials (the fourth serial port) or do I need to write something into the code in order to make sure it debugs?

CodenameAter: No, that explains why I don't receive any data from at serial. However, as of now I am actually running only three bowls so why would that impact what the other three serials ports receive and send?

In regards to a fourth serial. Robin suggested leaving one of the four serials free for debugging. I am not sure exactly what that entails. Do I simply just not use one of the four serials (the fourth serial port) or do I need to write something into the code in order to make sure it debugs?

Serial (i.e. NOT Serial1, Serial2 or Serial3) is the serial port that communicates with the PC using the USB connection. Don't use it for anything else until you have everything else working properly.

When you say "what the data should look like" COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:27, I think you are telling us what you want the output from your Arduino to look like.

But before you get to that you need to know what is actually sent to the Arduino. Many things send a linefeed or carriage return (or both) to mark the end of a message. Better programs also use a special character to mark the start of a message. What system is your device using when it sends data TO the Mega.

By the way, I don't think you have told us what the sending devices are - it would help to know.

...R

Robin2: But before you get to that you need to know what is actually sent to the Arduino. Many things send a linefeed or carriage return (or both) to mark the end of a message. Better programs also use a special character to mark the start of a message. What system is your device using when it sends data TO the Mega.

By the way, I don't think you have told us what the sending devices are - it would help to know.

Device? I am sorry but I am not sure what you mean. The Mega sends data to the Xbee Shield which is then sent to a receiving antenna on my PC.

What sending devices? I thought the Mega is using the Xbee Shield to send to the data to my PC. Data only flows in one direction: RFID Antenna --> RFID Module --> Mega --> Xbee Shield --> Receiving XBee antenna --> XCTU on my PC

Maybe you can specify on this list of devices, which you need more information on.

Thank you.

CodenameAter: Device? I am sorry but I am not sure what you mean. The Mega sends data to the Xbee Shield

I thought the whole problem is because the Mega is trying to receive data on the serial ports from 3 devices so that it can send the data onwards to the PC.

A simple diagram showing the complete data flow would be helpful.

...R

Sorry for my slow replies. It is our busy season in the lab and we’re working 10-15 hours a day, didn’t have a chance to create some sort of schematic. Anyway, here it is. Probably looks like a grade 5 put it together but it’s the best I could do. I only drew it with one antenna however, each Mega has four antennas attached to it, each attaching to RX ports 0, 1, 2 or 3.

To reiterate the problem. Data leaving the outbound antenna on the Xbee Shield should look like:

COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:27,
COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:38,
COOP-04191795B66,N/A,Food_1,N/A,5/27/2016,20:41:40,
COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:42,
COOP-04191795B66,N/A,Food_1,N/A,5/27/2016,20:41:43,

However, what shows up in XCTU or another COM port monitoring software is something like:

COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:27,COOP-041917D3522,N/A,Food_3,N/A,5/27/2016,20:41:38,
20:41:40,
20:41:42,
COOP-04191795B66,N/A,Food_1,N/A,5/27/2016,20:41:43,

I am sure you will have questions so please ask away.

RFID Design.pdf (197 KB)

CodenameAter: I am sure you will have questions so please ask away.

Thanks for the picture. It is pretty much what I imagined.

Now go back and read Replies #6 and #11 and provide the data I requested. That is, the data coming FROM the RFID unit to the Mega.

And start with a single RFID unit so you can view debug data on the Serial Monitor. Then move to 2 RFID units.

...R

Not sure what you mean. The only data that comes from the RFID Module to the Mega is a 10 digit code (ex. 041917AEDCC). I am not sure what you mean by start & end markers. An example of the code used to read the 10 digit code & apply the necessary labels is provided below.

If this isn’t what you are looking for, can you be more specific? Thank you.

void readRfidPort1() { //1st Pin (RX = pin 0 & TX = pin 1)
  if(Serial.available() > 0) { // if data available from reader
    if((val = Serial.read()) == 2) { // check for header
    bytesread = 0;
    while(bytesread<10) { // read 10 digit code
      if( Serial.available() > 0) {
        val = Serial.read();
          if((val == 2)||(val == 3)) { // if header or stop bytes before the 10 digit reading
          break; // stop reading
          }
          code[bytesread] = val; // add the digit
          bytesread++; // ready to read next digit
      }
    }
    if(bytesread == 10) { // if 10 digit read is complete
    //codeToSd();
    //timeToSd();
    Serial.print("RFID Aviary-Tag: ");
    Serial.print(Avy);
    Serial.print("-");
    Serial.println(code);
    Serial.print("No Data: ");
    Serial.println(BkN);
    Serial.print("Location & Location Type: "); 
    Serial.print(P0_L);
    Serial.print(" | ");
    Serial.println(P0_T);
    Serial.print("Writing Time to File: ");  
    digitalWrite(13, HIGH); //indicate it by glowing LED at pin 13
        delay(1500); // delay 1.5 seconds
        dataFile.print(Avy);
        dataFile.print("-");
        dataFile.print(code);
        dataFile.print(",");
        dataFile.print(BkN);
        dataFile.print(",");
        dataFile.print(P0_L);
        dataFile.print(",");
        dataFile.print(P0_T);
        dataFile.print(",");
            XBee.print(Avy);
            XBee.print("-");
            XBee.print(code);
            XBee.print(",");
            XBee.print(BkN);
            XBee.print(",");
            XBee.print(P0_L);
            XBee.print(",");
            XBee.print(P0_T);
            XBee.print(",");            
    timeToSd();
    dataFile.flush();
    digitalWrite(13, LOW); // glow off LED
    }
    bytesread = 0;
    }
  
} 
}

I can't really understand the context of the code in Reply #16.

Do you mean that you have (or plan to have) similar code reading data arriving at each of the Mega's hardware serial ports?

I don't understand why you are using Serial (rather than Serial1, Serial2 or Serial3) for your test code.

In the piece of code you have posted you are reading data from the RFID module and then sending a whole lot of stuff back to it - that makes no sense (at least to me).

I think I have already recommended that you keep Serial free for debug messages while you are developing the system.

You say your RFID module sends a 10 character (not digit!) code. Does it send a newline or carriage-return character after that so that you know that the full message has arrived? Counting is NOT reliable because you could be counting the last 3 chars in one message and the first 7 in the next.

Assuming it does use a newline character then just use the second example in Serial Input Basics to receive the data. Receive the data from all of your RFID modules and only then send the data using the XBee.

...R