Help with weird behavior when using Java(rxtx) for USB serial communication.

Hello!

During my current project i have hit a kind of wall. When trying to communicate between a java program(Using rxtx 64bit) on Windows 8(64 bit) and a Arduino UNO Rev 3, it periodically halts.
And with halts, i mean it takes the program about 600ms to make a "handshake"(Five bytes from the computer, one back from the arduino) at a baud rate of 115200 bits/second.

Another thing which i have noticed happens is that the Arduino's RX(in)/TX(out) serial light light up in an odd fashion. Without errors, the lights flash as they should, but when it halts, the RX light remains on, while the TX light blinks at the speed of the responses.

NOTE: The system is supposed to be able to use the full baud rate for communication and this is a way of testing it, basically a form of stress test.

NOTE 2: The handshakes might not be that efficient / to often, but this is by design, at least while debugging.

This is an output log i have created to demonstrate the issue.
It's build up with: "NUMBER_OF_BYTES_SENT: (I_IN_ITERATION_OF_MAX_60) = NUMBER_OF_MS_IT_TOOK_TO_PERFORM_HANDSHAKE"
and a line("-------------") when the problem temporarily resolves itself.
ex:

130735: (46) = 503ms
130740: (47) = 606ms
----------------------
130780: (5) = 606ms

Full logs that follow the same log pattern. They might not be that interesting but show some kind of error pattern:
https://dl.dropboxusercontent.com/u/39389013/Debug/1.txt
https://dl.dropboxusercontent.com/u/39389013/Debug/2.txt

The test code running on the Arduino is:

//#define serialDataRate 9600
//#define serialDataRate 38400
#define serialDataRate 115200

int last;
int rd;
long lastTime = 0;

void setup() {
  Serial.begin(serialDataRate);
  last = 0;
  lastTime = millis();
}

void loop() {
    while (Serial.available() >= 5) {
      lastTime = millis();
      for(int i = 0; i < 5; i++){
        rd = Serial.read();
        if(millis() - lastTime > 100)
          Serial.write(2); // Never happens? 
        last = rd;
      }
      Serial.write(1); // Handshake
    }
}

Demo code running on the computer:

package default;

import java.io.IOException;
import java.util.concurrent.CountDownLatch;

public class MainController {

    CountDownLatch latch = new CountDownLatch(1);
    SerialController sc;

    public MainController() {
        sc = new SerialController();
        SerialEventListener serialEvent = new SerialEventListener() {
            public void Recived(int data) {
                recivedPacket(data);
            }
        };
        sc.setEventListener(serialEvent);
        sc.initialize();
        System.out.println("Serial baud rate: " + sc.serialPort.getBaudRate());
    }

    public void recivedPacket(int data) {

        switch (data) {
        case 1:
            latch.countDown();
            // System.out.println("Hand shake");
            break;
        case 2:
            System.out.println("Error!");
            break;
        default:
            System.out.println("Recived other:" + data);
            break;
        }
    }

    int bt = 0;
    long lastTime = 0;
    boolean divide = false;
    byte[] buffer = new byte[5];

    public void update() {
        lastTime = System.currentTimeMillis();
        for (int i = 0; i < 60; i++) { // this is used since my real project,
                                       // when under maximum capacity needs to
                                       // be able to send 60 packets of data
                                       // every "frame".

            for (int itt = 0; itt < 5; itt++) { // Create a packet of 5 bytes.
                buffer[itt] = (byte) ((bt++) % 255); // Create a buffer to send.
                                                     // Sends an array of bytes
                                                     // in numerical order, 0 to
                                                     // 255, and then back to 0.
            }
            try {
                sc.write(buffer);
            } catch (IOException e1) {
                e1.printStackTrace();
            } // Using rxtx, writes the array to the serial connection.
            try {
                latch.await(); // Await the handshake
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            if (System.currentTimeMillis() - lastTime > 10) {
                System.out.println(bt + ": (" + i + ") = " + (System.currentTimeMillis() - lastTime) + "ms");
                divide = true;
            } else if (divide) {
                System.out.println("-------------------");
                divide = false;
            }
            lastTime = System.currentTimeMillis(); // Update the time so that
                                                   // only the issues are
                                                   // logged.
        }
    }

    public static void main(String[] args) throws Exception {
        MainController mc = new MainController();
        Thread.sleep(4000); // Wait 4 seconds for the connection to establish

        while (true) {
            mc.update();
            Thread.sleep(50); // Wait time, can be lower, but this seems like a good place to start when debugging. 
        }
    }
}

SerialController.java (sc). Modified version of a file that I've taken taken from an example on how to set up a connection:
https://dl.dropboxusercontent.com/u/39389013/Debug/SerialController.java
It also requires the rxtx library: RXTX for Java.

I'm sorry for not having a full project to share, but this should be enough to see the problem in action.

This is a problem I've been scratching my head at for hours, but with nothing to show for, so help with find a fix to this problem is deeply appreciated!

Thanks in advance.

Best regards
Ledge.

I wonder if you need a more sophisticated interface. For example you start reading if there are at least 5 bytes, but you only read 5. A buffer may be filling up. (Life is too short to read Java code, sorry).

The approach in the demo I wrote here Demo of PC-Arduino comms using Python - Interfacing w/ Software on the Computer - Arduino Forum may be of interest.

I designed the PC program using JRuby (which uses the JVM) and RxTx and have recently switched to JSSC as the Java serial library, but both worked and I had no trouble with either at 115200 baud. I have another JRuby/Arduino project where throughput is important.

...R

Robin2, thank you for the reply!

I wonder if you need a more sophisticated interface. For example you start reading if there are at least 5 bytes, but you only read 5. A buffer may be filling up. (Life is too short to read Java code, sorry).

I've actually thought about this earlier, but the current Java code sends 5 bytes, then waits for the Arduino to read those bytes and respond, I can't rely see why the buffer would fill up. (Not the most efficient, but it should be reliable)

Extracted from the debug log

...
89465: (12) = 270ms
-------------------
130450: (49) = 301ms
...

Since it takes about 90 000 bytes to be sent before it starts to halt, then another 40 000, a buffer overflow doesn't seem probable.

The approach in the demo I wrote here Demo of PC-Arduino comms using Python - Interfacing w/ Software on the Computer - Arduino Forum may be of interest.

I looked into this, but it seemed like a lot of extra bandwidth is used. It's probably the best way of doing it if reliability is the priority, but my system doesn't require the data to be correct a 100% of the time. It reads the "package" of 5 bytes, checks if the data is valid. (First number, currently 255, is a check which is not allowed in any other part of the package. ) If not, discard it, or if enough information is present, asks for the computer to resend the packet. The data is continuously updated, so a lost packet every once in a while doesn't do much.

I designed the PC program using JRuby (which uses the JVM) and RxTx and have recently switched to JSSC as the Java serial library, but both worked and I had no trouble with either at 115200 baud. I have another JRuby/Arduino project where throughput is important.

I will try implementing the JSSC to see if the error lies within the build of rxtx i'm using or elsewhere.

I'll tell you how it goes.

L.

Thanks again Robin2!

I've changed the serial controller from RxTx to JSSC and had it running the "stress test" for a quarter of an hour without an issue, where it would normally fail after just under a minute.

Seems like the RxTx binaries are a bit out of date. Latest build iv'e found are from 2008, so it's good to know that it was the reason the program halted.

L.

That's interesting to know.

My system just adds two extra bytes (a startmarker and an endmarker) if you don't need to include the number of bytes sent (if it's always 5).

By the way the USB system isn't at it's best if you only send small amounts of data. It only starts sending stuff once per millisecond (as far as I recall). If you are interested let me know and I will try to find a link to an explanatory paper on the subject.

...R

That would be great!

But i'm a bit doubtful of the USB interface only being able to send information every millisecond, since my code, which is closely resembles the example code, is able to transmit approximately 100000 bits/ 100 Kb a second without any problem, which is close to the maximum of 115 200 bits.

Another cause of this might be that the WRITE command actually writes to a buffer and not directly to the USB device, which i think is quite reasonable.
But no, since I wait for a handshake after every send command.

Still, if you know about any good paper on this subject, I would be happy to get my hands on it!

L.

Maybe I have promised more than I can deliver. It's a long time since I read about USB. I think this document is the one I used Embedded USB Design By Example by John Hyde

If not, look at other documents on FTDIs website. They are as expert as anyone in this field.

I also have a PDF with the USB 2.0 specifications which is long and tedious.

...R

At first glance, it seems like an interesting book/paper. I'll take a more thorough look at it this evening when I've got the time.
As you say, specification sheets are probably not the most interesting way to go, even if it is the backbone of the system.

You've been of great service and I hope your own future projects go well!

L.