Serial Monitor Distortion from Python/PySerial?

Projects Interfacing w/ Software on the Computer
1

I have encountered an error that has been seriously bothering me, and I have been unable to find a workaround. I have a python program that utilizes PySerial to import serial port values from a pulse oximeter. The problem is, when the function ser.readline() is invoked (in other words, when the python program is told to read values from the Arduino's serial monitor, the Arduino's serial values become distorted and the program returns a value unpack error.

In the python program, the following is called:

data_in = ser.readline()
print data_in
data_in = data_in.strip('\n')
bpm,sp02 = data_in.split(",")

The Arduino program is as follows:

#include <PinChangeInt.h>
#include <eHealth.h>

int cont = 0;

void setup() {
Serial.begin(115200);
eHealth.initPulsioximeter();

PCintPort::attachInterrupt(6, readPulsioximeter, RISING);
}

void loop() {

char buffer[32]; // make sure buffer is large enough
sprintf(buffer,"%d,%d \n",eHealth.getBPM(),eHealth.getOxygenSaturation());
Serial.print(buffer);
delay(500);

}

//=========================================================================
void readPulsioximeter(){

cont ++;

if (cont == 50) { //Get only of one 50 measures to reduce the latency
eHealth.readPulsioximeter();
cont = 0;
}
}

So, the serial monitor is outputting values like this:

67,95
66,95
67,96

and so on.

But only when ser.readline() is invoked, the values become skewed and are unable to be unpacked by the split(',') function. In the following photos (1) and (2) you can see the distortion of the values right when the ser.readline() is called.

How can I re-word the python OR Arduino program in such a way to circumvent this distortion and allow the values to be split and unpacked without any errors? Any help is appreciated! This problem has been bothering me for weeks!

2

Does the Arduino program work properly when it outputs to the Serial Monitor ?

A common problem when people write PC programs is that they do not take account of the fact that the Arduino resets when the PC program opens the serial port. It should only open the serial port at the start and then keep it open until completely finished with the Arduino. As you have only posted a few lines of your Python program I have no idea if that is your problem.

This Python-Arduino demo may give you some ideas.

...R

3

How can I re-word the python OR Arduino program in such a way to circumvent this distortion and allow the values to be split and unpacked without any errors? Any help is appreciated! This problem has been bothering me for weeks!

If you are using Python AND the Serial Monitor application to read serial data AT THE SAME TIME, as your photos imply, interference IS to be expected. That is what you are seeing, not distortion.

4

PaulS:
If you are using Python AND the Serial Monitor application to read serial data AT THE SAME TIME, as your photos imply, interference IS to be expected. That is what you are seeing, not distortion.

Yes and no. Yes, the data is more prone to be skewed when both are open. However, the error still occurs when only the program is being run.

5

Robin2:
Does the Arduino program work properly when it outputs to the Serial Monitor ?

A common problem when people write PC programs is that they do not take account of the fact that the Arduino resets when the PC program opens the serial port. It should only open the serial port at the start and then keep it open until completely finished with the Arduino. As you have only posted a few lines of your Python program I have no idea if that is your problem.

This Python-Arduino demo may give you some ideas.

...R

Hi, thanks for your reply. The Arduino does indeed output perfect data when the ser.readline() function is not called. By perfect, I mean I see one line after another, and just two values separated by a comma. Being generated like that all day. But the second readline is invoked, it skews that data. Indeed, the port is opened at the start of my program, and a short while after, readline is called. Please take a look at this program, in addition to the Arduino code and please help me understand why this is occurring. My background is in mechanics, so I am having a difficult time with computing. Your help is greatly appreciated.

Python program:
import serial
import time
import pylab
import matplotlib.pyplot as plt
import numpy as np
import sys
import os
import csv
from pylab import *
from collections import OrderedDict

#time load
timestr = time.strftime("%Y_%m_%d")
#spacer
spacer = "_"
spacer2 = " "
#user inputs name
last_name = raw_input('Enter last name: ')
first_name = raw_input('Enter first name: ')
name = last_name + first_name
file_name = name + spacer + timestr
file_name_str = file_name+'.csv'
#viewable names
viewable_name = first_name + spacer2 + last_name
viewable_filename = viewable_name + spacer + timestr

#change directory on computer to data folder
path = '/home/pi/Desktop/pulseox/data'
os.chdir(path)
text_file = open(file_name, "w")
full_path = path + "/" + file_name_str

#establish serial connection with ACM0
ser = serial.Serial('/dev/ttyACM0', 115200)

#establish variables
xL = [ ]
bpmL = [ ]
spo2L = [ ]

xL_short = [ ]
bpmL_short = [ ]
spo2L_short = [ ]

array_data = xL, bpmL, spo2L
array_data_short = xL_short, bpmL_short, spo2L_short

#graph attributes
f, (ax1, ax2) = plt.subplots(1, 2, sharey = True)
plt.ion()
#axes
ax1.yaxis.grid()
ax2.yaxis.grid()

#declare zeroth values
x = 0
x_short = 0
ser.readline()

while True:

data_in = ser.readline()
print data_in
data_in = data_in.strip('\n')
bpm,spo2 = data_in.split(",")
bpm_short ,spo2_short = bpm, spo2

#convert string vals to float
x = float(x)
bpm = float(bpm)
spo2 = float(spo2)

x_short = float(x_short)
bpm_short = float(bpm_short)
spo2_short = float(spo2_short)

#print to terminal
print "Time : %s" % (x)
~~ print "HR [BPM]: %s" % (bpm)~~
~~ print "SPO2 [%%]: %s" % (spo2) ~~
~~ print~~
#append vectors
~~ xL.append(x)~~
~~ bpmL.append(bpm)~~
~~ spo2L.append(spo2)~~
~~ xL_short.append(x_short)~~
~~ bpmL_short.append(bpm_short)~~
~~ spo2L_short.append(spo2_short)~~
#print values to plot
~~ ax1.plot(xL_short, bpmL_short, marker = 'o', linestyle = '-', color = 'red' ,label="Pulse Rate (BPM)")~~
~~ ax1.plot(xL_short, spo2L_short, marker = 'o', linestyle = '-', color="blue",label = "SPO2 (%)")~~
~~ ax2.plot(xL, bpmL, marker = 'o', linestyle = '-', color = 'red' ,label="Pulse Rate (BPM)")~~
~~ ax2.plot(xL, spo2L, marker = 'o', linestyle = '-', color="blue",label = "SPO2 (%)")~~

~~ plt.pause(0.1)~~
~~ time.sleep(0.05)~~
#legend
~~ handles, labels = plt.gca().get_legend_handles_labels()~~
~~ by_label = OrderedDict(zip(labels, handles))~~
~~ ax1.legend(by_label.values(),by_label.keys(),loc='lower right',fancybox = True, framealpha = 0.5)~~
~~ ax2.legend(by_label.values(),by_label.keys(),loc='lower right',fancybox = True, framealpha = 0.5)~~
#limit plot1 axis to 30 sec
~~ ax1.set_xlim([0,30])~~
#update time
~~ x = x+1~~
~~ x_short = x_short + 1~~
~~ if x_short >= 30:~~
~~ x_short = 0~~
~~ ax1.cla()~~
~~ xL_short = [ ]~~
~~ bpmL_short = [ ]~~
~~ spo2L_short = [ ]~~
#plt.title("Pulse [BPM] & SPo2 [%] v. Time ", fontsize = "16")
#plt.xlabel("Time ", fontsize = "14")
~~ #plt.ylabel("Pulse (red) [BPM] & SPo2 (blue) [%]", fontsize = "14")
 ax1.yaxis.grid()
 ax2.yaxis.grid()
#write to .csv~~
~~ with open(full_path, 'w') as f:
 writer = csv.writer(f)
 for t, b, s in zip(array_data[0], array_data[1], array_data[2]):
 writer.writerow([t, b, s])~~

6

By perfect, I mean I see one line after another,

See it where? You can NOT have two applications reading serial data from the same sorce, and expect them to not interfere.

Please take a look at this program

Why? I'm 99.99999% certain that you code does NOT look like that.

7

preacherperkins:
Please take a look at this program,

Post your program using the code button </> so it looks like this

...R

8

You can NOT have two applications reading serial data from the same sorce, and expect them to not interfere.

Why do you think that? That is probably only true if the two applications were trying to use the same com port on a computer.

9

That is probably only true if the two applications were trying to use the same com port on a computer.

Wouldn't "from the same source" imply that? It certainly excludes any other possibility as far as I'm concerned. YMMV.

10

You can NOT have two applications reading serial data from the same sorce, and expect them to not interfere.

Wouldn't "from the same source" imply that? It certainly excludes any other possibility as far as I'm concerned. YMMV.

The arduino (or pc) Tx pin doesn't really care what is attached as long as what is attached has the electrical properties of an rx pin. For debugging I have a servo controller attached to the arduino tx pin, and the arduino serial chip is also attached to the arduino tx pin. When the arduino sends a command out the tx pin,it goes to the servo controller, and also displays on the arduino serial monitor. Aka, two devices reading serial data from a single source.

11

Wonderful. That is NOT what OP is doing, however, so it is irrelevant.

12

PaulS:
Wonderful. That is NOT what OP is doing, however, so it is irrelevant.

I am commenting on the bad info you posted. 8)

13

I posted NO bad info, you idiot. OP is trying to use two applications on the PC to read from ONE serial port. That WILL ABSOLUTELY cause corruption of data.

It is nowhere near the same as sending data to two different devices that each make a copy of the data.

14

OP is trying to use two applications on the PC to read from ONE serial port. That WILL ABSOLUTELY cause corruption of data.

Well, perhaps you should say that normally two applications on the same pc cannot use the same serial port at the same time.

15

PaulS:
I posted NO bad info, you idiot. OP is trying to use two applications on the PC to read from ONE serial port. That WILL ABSOLUTELY cause corruption of data.

It is nowhere near the same as sending data to two different devices that each make a copy of the data.

PaulS, I am trying to learn as much as I can about the device and my program. I am grateful for any help I can get from the Arduino community, but your antagonizing does not belong here so please take it elsewhere.

16

Hi all,

Thank you all for your input.

When I execute the program, it is the only process being executed on my computer. In other words, only terminal on my RBP is running. I'd say the serial line being inputted becomes skewed 80% of the time. The other 20% of the time, it works. Perhaps due to chance. It is a mystery to me.

Those who posit that having two programs running will contribute to the interference are correct. Indeed, when the program runs, and I open the Arduino IDE (not even the serial monitor, just the IDE), the program returns an unpack error due to a skewed readline.

I am inclined to believe that my python program is sound. (I apologize for not posting it in the proper format). PySerial is not an uncommon module and usually produces good results. On the other hand, I am not as familiar with programming Arduinos as I am with making a python program. It is for this reason I ask the Arduino community if my Arduino program can somehow be written to circumvent the error. All help and attention received by my inquiry is greatly appreciated.

import serial
import time 
import pylab
import matplotlib.pyplot as plt
import numpy as np
import sys
import os
import csv
from pylab import *
from collections import OrderedDict

#time load
timestr = time.strftime("%Y_%m_%d")
#spacer
spacer = "_"
spacer2 = " "
#user inputs name
last_name = raw_input('Enter last name:  ')
first_name = raw_input('Enter first name: ')
name = last_name + first_name
file_name = name + spacer + timestr
file_name_str = file_name+'.csv'
#viewable names
viewable_name = first_name + spacer2 + last_name
viewable_filename = viewable_name + spacer + timestr

#change directory on computer to data folder
path = '/home/pi/Desktop/pulseox/data'
os.chdir(path)
text_file = open(file_name, "w")
full_path = path + "/" + file_name_str

#establish serial connection with ACM0
ser = serial.Serial('/dev/ttyACM0', 115200)

#establish variables
xL = [ ]
bpmL = [ ]
spo2L = [ ]

xL_short = [ ]
bpmL_short = [ ]
spo2L_short = [ ]

array_data = xL, bpmL, spo2L
array_data_short = xL_short, bpmL_short, spo2L_short

#graph attributes
f, (ax1, ax2) = plt.subplots(1, 2, sharey = True)
plt.ion()
#axes
ax1.yaxis.grid()
ax2.yaxis.grid()

#declare zeroth values
x = 0
x_short = 0
ser.readline()

while True:  
          
    data_in = ser.readline()
    print data_in
    data_in = data_in.strip('\n')
    bpm,spo2 = data_in.split(",") 
    bpm_short ,spo2_short = bpm, spo2
  
#convert string vals to float
    x = float(x)
    bpm = float(bpm)
    spo2 = float(spo2)

    x_short = float(x_short)
    bpm_short = float(bpm_short)
    spo2_short = float(spo2_short)

#print to terminal
    print "Time [s]: %s" % (x)
    print "HR [BPM]: %s" % (bpm)
    print "SPO2 [%%]: %s" % (spo2)  
    print 

#append vectors
    xL.append(x)
    bpmL.append(bpm)
    spo2L.append(spo2)

    xL_short.append(x_short)
    bpmL_short.append(bpm_short)
    spo2L_short.append(spo2_short)

#print values to plot
    ax1.plot(xL_short, bpmL_short, marker = 'o', linestyle = '-', color = 'red' ,label="Pulse Rate (BPM)")
    ax1.plot(xL_short, spo2L_short, marker = 'o', linestyle = '-', color="blue",label = "SPO2 (%)") 

    ax2.plot(xL, bpmL, marker = 'o', linestyle = '-', color = 'red' ,label="Pulse Rate (BPM)")
    ax2.plot(xL, spo2L, marker = 'o', linestyle = '-', color="blue",label = "SPO2 (%)") 
    
    plt.pause(0.1)
    time.sleep(0.05)

#legend
    handles, labels = plt.gca().get_legend_handles_labels()
    by_label = OrderedDict(zip(labels, handles))
    ax1.legend(by_label.values(),by_label.keys(),loc='lower right',fancybox = True, framealpha = 0.5)
    ax2.legend(by_label.values(),by_label.keys(),loc='lower right',fancybox = True, framealpha = 0.5)

#limit plot1 axis to 30 sec
    ax1.set_xlim([0,30])

#update time
    x = x+1
    x_short = x_short + 1

    if x_short >= 30:
        x_short = 0
        ax1.cla()
        xL_short = [ ]
        bpmL_short = [ ]
        spo2L_short = [ ]
        #plt.title("Pulse [BPM] & SPo2 [%] v. Time [s]", fontsize = "16")
        #plt.xlabel("Time [s]", fontsize = "14")
        #plt.ylabel("Pulse (red) [BPM] & SPo2 (blue) [%]", fontsize = "14")
        ax1.yaxis.grid()
        ax2.yaxis.grid()

#write to .csv
    with open(full_path, 'w') as f:
        writer = csv.writer(f)
        for t, b, s in zip(array_data[0], array_data[1], array_data[2]):
            writer.writerow([t, b, s])
17

Your code does not check if the Arduino has completed its reset process.

Your code does not close the serial port when it is finished with the Arduino.

Did you study the code in the link I gave you in Reply #1

...R