Why is USBSerial so slow for duplex communication?

Using Arduino Interfacing w/ Software on the Computer
#1

Hi, I have a project where I control the Arduino from a RasPi. A python program listens for incoming messages from an MQTT queue and dispatches commands to the Arduino. Since I am using Servos, I need to wait for the commands to complete before new commands are issued.

Here's a YouTube video showing how it worked before I used GPIO: The movements are jerky and it takes ages from I throw a switch in Minecraft until the eyes in the skull turn off: The real skull sees you in minecraft! - YouTube

When I used USBSerial, the roundtrip time was a full second, no matter how simple the command was (like turning on a LED). I asked around, and was told to use GPIO. I then purchased a logic converter from Adafruit and used the SoftSerial library on the Arduino. The speed increase was dramatic! Now the roundtrip time can be measured in miliseconds. Why is this so?

The code is almost unchanged, I just had to adjust for the limitations in the SoftSerial library compared to Serial object.

I have extracted the before and after code in two gists:

Python: The python/raspi side in arduino communication · GitHub
Arduino: Arduino side with Serial and SoftSerial · GitHub

If you use softserial make sure to protect the Pi with a logical converter

#2

Without spending more time than I am prepared to do I can't figure out from your code what "round trip" you are having a problem with.

I haven't found any problem using the normal USB serial connection - as far as I recall it worked up to 1 megabaud in a test - apart from the normal USB latency, but that's only about 1 millisecond.

I have a working project that transmits and receives 5 times per second quite reliably.

Perhaps all of the delay()s in your code are the problem?

Have you experimented with a simple piece of code (in both Python and Arduino) that tests the round trip performance with no other distractions?

...R

#3

Hi, thanks for answering.

Roundtrip: Time it took from the command was sent to the Arduino until I got a reply back. Before GPIO it was > a second. (see line 31 onwards in dispatcher.py

After changing the code to use GPIO it was like a half millisecond. No other changes were made, (Yes I have added some more functionality later, but not before I measured the new results)

Some delays are there to allow the servos to move, btw. Other delays may or may not be necessary, byt again: they are the same as before.

Yes I tested the code without other distractions like the MQTT queue, and the results were the same.

#4

sisomm:
Roundtrip: Time it took from the command was sent to the Arduino until I got a reply back. Before GPIO it was > a second. (see line 31 onwards in dispatcher.py

I was hoping you would describe the steps involved in the round trip so I wouldn't have to read the code?

...R

#5

OK:On the Python I send simple commands to the Arduino and wait for response. The loop below performs at least 1000 times faster with SoftSerial on the other side. No other changes were made in the communication, bar using a '|' as a command terminator

arduino.write(command)
response=''
ack=False
while not ack:
response=arduino.readline()
if (len(response)>0):
ack=True

end=time.time()
print('Response to {} took {:G} millis'.format(response.strip(),(end-start)*1000))

#6

Actually, before I used Python it was blazingly fast. I just used shell commands on the mac and piped things in and out of the queue. this didn't work for raspi, and besides i needed to wait for ack. But it was fast!

from MQTT:
$mosquitto_sub -t /arduino/1/incoming -v | gawk -f transform.awk > /dev/ttyACM0

from Arduino:
$tail -f /dev/ttyACM0 | mosquitto_pub -t /arduino/1/status -l?

#7

I think this is a problem with the raspberryPi end of things. It is the Linux that slows things down and the Python also adds a lot to the delays.

I have a project where I am sending bytes at a rate of 40 per second from the Arduino to the Pi through the USB serial port and while the Pi keeps up sometimes there is 4K of data in the buffer waiting to be processed.

#8

I should have mentioned in my Reply #1 that I am using JRuby on a Linux netbook. I would expect Python to perform as well as JRuby.

If you can post a short Arduino sketch that demonstrates your problem I will try it out.

...R

#9

I have extracted the before and after code in two gists:

Python: The python/raspi side in arduino communication · GitHub
Arduino: Arduino side with Serial and SoftSerial · GitHub

Oh and thanks for taking the time!
If you use softserial make sure to protect the Pi with a logical converter

#10

I can't open either of your links. Can you post the code in the forum?

...R

#11

OK. Here's the Python code:

## How it was with Serial
 
        if(not commands.empty()):
            command=commands.get()
            if(args.verbosity>0):
                print("DISPATCHER: sending to Arduino: "+command)
            start=time.time()
            arduino.write(command)
 
            # wait until we get OK back
            response=''
            ack=False
            while not ack:
                response=arduino.readline()
                if (len(response)>0):
                    ack=True
 
            end=time.time()
            print('Response to {} took {:G} millis'.format(response.strip(),(end-start)*1000))
 
 
## How it is with SoftSerial
 
    while True:
        if(not commands.empty()):
            command=commands.get()
            if(args.verbosity>0):
                print("DISPATCHER: sending to Arduino: "+command)
            start=time.time()
            arduino.write(command+'|')
 
            # wait until we get OK back
            response=''
            ack=False
            while not ack:
                response=arduino.readline()
                if (len(response)>0):
                    ack=True
 
            end=time.time()
            print('Response {} to {} took {:G} millis'.format(response,command,(end-start)*1000))
#12

And here's the Arduino code:

// How it was with Serial:
      
      // *** Receive ***
      
        byteCount = -1;
        byteCount =  Serial.readBytesUntil('\n',buffer,bSize);  
      
        if (byteCount  > 0) {          // Really simple parsing
          strcpy(command,strtok(buffer,","));
      
      /*
          Then perform stuff...
      */
      
      // *** Send *** 
      
         }
          Serial.println(theCommand);
    
    
// NOW THIS IS WHAT IT LOOKS LIKE
 
 
        // *** Receive ***
        
        SoftwareSerial mySerial(3, 2); // RX, TX
        
        byteCount = 0;
          int finished=0;
          while(finished==0){
            char c = mySerial.read();
            if(c>=0){
              if((c=='|') || byteCount==(bSize-1)){  
                finished=1;
                buffer[byteCount]=0;
              } else {
                buffer[byteCount++]=c;
              }
            }
          }
        
        
          if (byteCount  > 0) {          // Really simple parsing
        
        /*
            Then perform stuff...
        */
        
        // *** Send ****
        
            mySerial.print(theCommand);
            mySerial.print('\n');
#13