Arduino and PC two-way using PySerialTransfer

Schnell · February 1, 2020, 9:47pm

Hi,

I have some trouble getting two-way communication working between an Arduino UNO and a Win10 PC through USB. Trying to use the PySerialTransfer library by Power_Broker.

Code:

#include "SerialTransfer.h"

SerialTransfer myTransfer;

bool mustanswer = false;

void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop()
{
  if(myTransfer.available())
  {
    mustanswer = true;
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
  
  if(mustanswer == true)
  {
    myTransfer.txBuff[0] = 'h';
    myTransfer.txBuff[1] = 'e';
    myTransfer.txBuff[2] = 'y';
    
    myTransfer.sendData(3);
    delay(100);
  }

}

The code here is just to debug. I started out with the standard example that I couldn't get working either. The python part is still the standard example (changed port).

I can receive data on the PC from the Arduino (when sending data is not conditional), but it seems something goes wrong with the PC sending to Arduino. Specifically I find that "myTransfer.available()" never becomes true - resulting in nothing being returned to the PC. I can see the TX LED blinking "once" when I try to send a few bytes from the PC, either from python or the IDE serial monitor. I don't receive any answer either in python or serial monitor (2 separate tests, as they can't both have the port).

As expected, if I manually set "mustanswer = true" out in the main loop I receive "hey" in python - and if looking in the IDE serial monitor it keeps repeating "~⸮hey⸮".

I also found that if I change the code from:

...
if(myTransfer.available())
...

Into:

...
if(Serial.available() > 0)
...

And send a few characters from the IDE serial monitor then I also get a result back, constantly repeating "~⸮hey⸮". I don't get any response in python if trying the same with the script though, it just hangs.

I also have a Mega that I tested with, no difference.

The basic plan is that a piece of software on the PC must interface with an Arduino controlled turntable through this python script/driver - when the driver has send the command the Arduino executes and a few seconds later responds with a result to the driver - if all went well the driver will let the PC software continue. Using PySerialTransfer might not be necessary, but it seemed like a nice way of getting there.

I hope someone can assist. Thanks.

Power_Broker · February 2, 2020, 1:56am

Python:

from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM15')
    
        link.txBuff[0] = 'h'
        link.txBuff[1] = 'i'
        link.txBuff[2] = '\n'
        
        link.send(3)
        
        while not link.available():
            if link.status < 0:
                print('ERROR: {}'.format(link.status))
            
        print('Response received:')
        
        response = ''
        for index in range(link.bytesRead):
            response += chr(link.rxBuff[index])
        
        print(response)
        link.close()
        
    except KeyboardInterrupt:
        link.close()

Arduino (Teensy 3.5):

#include "SerialTransfer.h"

SerialTransfer myTransfer;

bool mustanswer = false;

void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);
}

void loop()
{
  if(myTransfer.available())
  {
    mustanswer = true;
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
  
  if(mustanswer == true)
  {
    myTransfer.txBuff[0] = 'h';
    myTransfer.txBuff[1] = 'e';
    myTransfer.txBuff[2] = 'y';
    
    myTransfer.sendData(3);
    delay(100);
  }
}

Output (Python):

Response received:
hey

Seems to work on my end - note that the example Python script exits once it gets its first valid response from the Arduino.

Power_Broker · February 2, 2020, 2:32am

If you want an example Python script that continuously communicates with the Arduino, use this (tested and verified):

from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM15')
    
        while True:
            link.txBuff[0] = 'h'
            link.txBuff[1] = 'i'
            link.txBuff[2] = '\n'
            
            link.send(3)
            
            while not link.available():
                if link.status < 0:
                    print('ERROR: {}'.format(link.status))
                
            print('Response received:')
            
            response = ''
            for index in range(link.bytesRead):
                response += chr(link.rxBuff[index])
            
            print(response)
        
    except KeyboardInterrupt:
        link.close()

Python Output:

Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey

Schnell · February 2, 2020, 6:35pm

Thanks Power_Broker.

I can't get it working. I tried the exact scripts you posted (except other COM port). I tried it on two different Win10 PCs and different OS patch levels. I tried python 2.7 and 3.7 on both PCs. Tried the original UNO and the knockoff Mega. I might try a VM with CentOS.

I can't help think that something is special with that Teensy. Anyone got it working with a normal Arduino/PC combo?

Edit: also tried different USB cables.

Power_Broker · February 2, 2020, 10:14pm

Schnell:
I can't help think that something is special with that Teensy. Anyone got it working with a normal Arduino/PC combo?

Doing some testing with one of my Megas and I seem to be getting similar results as you - interesting...I don't see any reason why different Arduinos would give different results, but I'll take a look into it.

Power_Broker · February 2, 2020, 10:27pm

Ok, so I think it's a timing issue. I got it to work using the following programs:

Python:

from pySerialTransfer import pySerialTransfer as txfer

if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM13')
    
        while True:
            link.txBuff[0] = 'h'
            link.txBuff[1] = 'i'
            link.txBuff[2] = '\n'
            
            link.send(3)
            
            while not link.available():
                if link.status < 0:
                    print('ERROR: {}'.format(link.status))
                
            print('Response received:')
            
            response = ''
            for index in range(link.bytesRead):
                response += chr(link.rxBuff[index])
            
            print(response)
        
    except KeyboardInterrupt:
        link.close()

Arduino:

#include "SerialTransfer.h"


SerialTransfer myTransfer;

bool mustanswer = false;


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);

  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
}


void loop()
{
  if(myTransfer.available())
  {
    mustanswer = true;
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }
  

  myTransfer.txBuff[0] = 'h';
  myTransfer.txBuff[1] = 'e';
  myTransfer.txBuff[2] = 'y';
  
  myTransfer.sendData(3);

  if(mustanswer)
  {
    mustanswer = false;
    
    digitalWrite(13, HIGH);
    delay(100);
    digitalWrite(13, LOW);
    delay(100);
  }
  else
    delay(200);
}

Arduino Output:
LED Blinks at 5Hz

Python Output:

Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey
Response received:
hey

Let me know what the results are for you and we can see if we can get the libraries to work with your real project.

Power_Broker · February 3, 2020, 10:15pm

After doing some thinking, I imagine the reason the original code you tried worked for the Teensy and not the Mega was because the Teensy doesn't reset once the COM port is opened by Python. Megas (and most other Arduinos) do, however, reset once a program opens its COM port. This makes a difference, because the example Python code immediately writes the first packet as soon as the COM port is open.

For the Teensy this didn't matter, but the Mega was busy bootloading when the packet from Python arrived. Because of this, the Mega didn't see the packet and never responded. Since the Python code blocks until it finds a response, it didn't try sending another packet.

Does that make sense?

Schnell · February 4, 2020, 7:36pm

Hi Power_Broker,

Power_Broker:
...
Does that make sense?

Yes it does actually. I think I can confirm that reset thing - takes around a second before the script(s) start doing something after the serial opening.

However I tried the code you posted. The results vary for me I have to say. Sometimes the LEDs blink and sometimes not. In any case, I still can't get it working two-way. It seems that myTransfer.available() keeps being "0". See example code below. The thing is I need this communication to be two-way and conditional: wait for a cmd -> execute stuff -> send back a response. I just never get a cmd it seems.

Arduino

#include "SerialTransfer.h"


SerialTransfer myTransfer;

bool mustanswer = false;


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);

  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
}


void loop()
{
  if(myTransfer.available())
  {
    mustanswer = true;
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }

  if(mustanswer)
  {
    myTransfer.txBuff[0] = 'h';
    myTransfer.txBuff[1] = 'e';
    myTransfer.txBuff[2] = 'y';
   
    myTransfer.sendData(3);
    mustanswer = false;
   
    digitalWrite(13, HIGH);
    delay(100);
    digitalWrite(13, LOW);
    delay(100);
  }
  else
    delay(200);
}

Python

from pySerialTransfer import pySerialTransfer as txfer

try:
	link = txfer.SerialTransfer('COM4')
	sleep(1)

	while True:
		link.txBuff[0] = 'h'
		link.txBuff[1] = 'i'
		link.txBuff[2] = '\n'
	   
		link.send(3)
	   
		while not link.available():
			if link.status < 0:
				print('ERROR: {}'.format(link.status))
		   
		print('Response received:')
	   
		response = ''
		for index in range(link.bytesRead):
			response += chr(link.rxBuff[index])
	   
		print(response)
   
except KeyboardInterrupt:
	link.close()

Nonetheless I would very much like to continue with the rest of this project (which is part of a much bigger project), so for now I have written a more simple serial transfer code, that does the job. It's crude and without CRC checks or anything, but it works (for now).

Note: it seems crucial that the Arduino gets to do a serial.println("stuff") before you continue.

Arduino

void setup() {
  Serial.begin(115200);
  Serial.println("A");
}

void loop() {

  if(Serial.available() > 6){
    for(int i = 0; i < 7; i++)
      Serial.write(Serial.read());
  }
}

Python

try:
	ser = serial.Serial('COM4', 115200, timeout=None)

	print(ser.readline().decode())
	sendstring = '+001234'
	for i in range(0,len(sendstring)):
		ser.write(sendstring[i].encode('utf-8'))
		
	while ser.inWaiting() < 7:
		sleep(.001)
	
	print(ser.read(7).decode())
	ser.close()
except KeyboardInterrupt:
	ser.close()

Should you by chance stumble upon the holy grail of solutions please do write and I'll check it out

Power_Broker · February 5, 2020, 12:57am

Schnell:
Should you by chance stumble upon the holy grail of solutions please do write and I'll check it out

The solution is either you're unlucky or have gremlins in your Arduino

In all seriousness, the real solution is that you just have to take into account some special considerations (as is needed when using any software).

Before giving up on the libraries completely, try this:

Python:

from time import sleep
from pySerialTransfer import pySerialTransfer as txfer


if __name__ == '__main__':
    try:
        link = txfer.SerialTransfer('COM13')
        
        link.open()
        sleep(5)
    
        while True:
            link.txBuff[0] = 'h'
            link.txBuff[1] = 'i'
            link.txBuff[2] = '\n'
            
            link.send(3)
            
            while not link.available():
                if link.status < 0:
                    print('ERROR: {}'.format(link.status))
                
            print('Response received:')
            
            response = ''
            for index in range(link.bytesRead):
                response += chr(link.rxBuff[index])
            
            print(response)
        
    except KeyboardInterrupt:
        link.close()

Arduino:

#include "SerialTransfer.h"


SerialTransfer myTransfer;

bool mustanswer = false;


void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);

  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
}


void loop()
{
  if(myTransfer.available())
  {
    mustanswer = true;
  }
  else if(myTransfer.status < 0)
  {
    Serial.print("ERROR: ");
    Serial.println(myTransfer.status);
  }

  if(mustanswer)
  {
    myTransfer.txBuff[0] = 'h';
    myTransfer.txBuff[1] = 'e';
    myTransfer.txBuff[2] = 'y';
  
    myTransfer.sendData(3);
    mustanswer = false;
  
    digitalWrite(13, HIGH);
    delay(100);
    digitalWrite(13, LOW);
    delay(100);
  }
  else
    delay(200);
}

Although I didn't touch the Arduino sketch, I added the following lines to the Python script after initializing the "link" object/class:

       link.open()
        sleep(5)

This opens the port and waits 5 seconds to allow the Arduino to completely reboot before trying to communicate with it. I've personally verified the above software works flawlessly with my Python and Mega. Let me know if it fixes things for you!

Tbh, I'm glad you reported your issue - it gives me really valuable feedback to constantly improve
my library code. Thanks!

Schnell · February 6, 2020, 11:26pm

I always wanted a gremlin

The code you have posted works fine here as well.

However the scenario for my project is a bit different - I need Python to send a command once, Arduino to receive and execute once, and finally return a response once. I tried converting the code into that scenario but I haven't succeeded.

It's a bit important in my project that the command arrives fast. The hardware execution itself (i.e. turning the table) will take around 3-4 seconds. After that the measurement starts and a few seconds later the turntable will be commanded to move again - repeated 10-15 times typically for a full measurement. I can make your posted code / the flow work faster by removing the sleep(5) and instead insert an initial transmit from the Arduino - setup phase below:

void setup()
{
  Serial.begin(115200);
  myTransfer.begin(Serial);

  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);

  myTransfer.txBuff[0] = '+';
  myTransfer.txBuff[1] = '+';
  myTransfer.txBuff[2] = '+';
 
  myTransfer.sendData(3);
}

My scenario is still a bit different of course, as mentioned.

Power_Broker:
Tbh, I'm glad you reported your issue - it gives me really valuable feedback to constantly improve
my library code. Thanks!

Happy to help!

Power_Broker · February 7, 2020, 1:12am

Schnell:
It's a bit important in my project that the command arrives fast. The hardware execution itself (i.e. turning the table) will take around 3-4 seconds. After that the measurement starts and a few seconds later the turntable will be commanded to move again - repeated 10-15 times typically for a full measurement.

That's totally doable with the libraries - I designed the libraries to send commands to Arduino-based RC airplanes with packet rates of over 50Hz.

Schnell:
I can make your posted code / the flow work faster by removing the sleep(5) and instead insert an initial transmit from the Arduino

The "sleep(5)" line is only executed in the Python script once during initialization and does not get called again. Since it's only called once during the Python script's startup, it won't slow down your communication at all once startup is complete.

Topic		Replies	Views
Is 2 way communication with PC via serial possible [UNO]? Interfacing w/ Software on the Computer	6	5145	May 6, 2021
pySerialTransfer Two Way Communication Issues Programming Questions	16	118	July 16, 2024
serial communication with pc using pySerial Programming Questions	3	725	May 6, 2021
communicate betwheen Arduino & PC fails - making clear to use pyserail correct Programming Questions	5	542	May 6, 2021
Problem with Pyserial Interfacing w/ Software on the Computer	5	8987	May 6, 2021

Arduino and PC two-way using PySerialTransfer

Related topics