Strange behavior when Serial Monitor is open

Using Arduino Programming Questions
#1

First, I apologize if this question is in the wrong place.

I am making a device which pulls a fiber optics thread from a spool and cuts and polishes the fiber. I’ve made a GUI in python where the user can send a length to be cut between 1"-6" with 0.01" tolerance. Essentially, the Python program sends the action (cut, polish, or both), the number of fibers to be cut and polished, the integer part of the length of the fibers, the number of decimal points in the fractional part, and then each decimal place. (This might seem overcomplicated, and if you have a better way of doing it please let me know.)

Problem: The program behaves exactly as it is supposed to behave with the serial monitor open. With the serial monitor closed I can only send the program lengths with a tolerance of 0.1. So for example, if I send the length 1.15" nothing happens, but if I send 1.1" the program works correctly.

First partial solution: Previously, I wasn’t able to send anything without the serial monitor open. I noticed the program restarted every time a serial connection was made. So I fixed this by adding (which was suggested in an older thread)

ser.dsrdtr=False
ser.setDTR(level=False)

Adding these lines allows me to send numbers of the form X.X but not X.XX.

Second partial solution: If I add a delay of 0.001 second in between ANY of my ser.write commands in Python, the program works correctly. You will see a few commented lines #time.sleep(.001) in my Python code. If I uncomment one of them, and it doesn’t matter which one, the program works without the serial monitor.

Why? What is happening? I assume that without the delay the Arduino is not receiving all of the information sent by the GUI. (Not sure how to test this idea since opening the serial monitor will change the behavior.) But what about having the serial monitor open allows this to happen correctly?

Here is part of my Python code (I removed the lines I thought were unnecessary):

# above this line is the GUI code
	def OnClose(self, event):
		action = 0	# 0 = do nothing, 1 = cut, 2 = polish, 3 = both
		# here is some code that sets the action from the user's input, removed to make more readable
		decimal, integer = math.modf(self.length.GetValue())
		decimalList = list(str(decimal))
		# send input to arduino
		ser.write(chr(int(action))) 					# send action
		#time.sleep(.001)
		ser.write(chr(int(self.number.GetValue()))) 	# send number of fibers
		#time.sleep(.001)
		ser.write(chr(int(integer))) 					# send length, int part
		#time.sleep(.001)
		ser.write(chr(int(len(decimalList))))			# send number of digits of frac
		time.sleep(.001)
		for i in range(2,len(decimalList)):				# send length, frac part
			ser.write(chr(int(decimalList[i])))
			#time.sleep(.001)
		ser.close()
		self.Close()
		
# search serial ports for arduino by VID:PID and SNR
# VID:PID and SNR for DFRduino RoMeo V1.1
# VID:PID = 2341:0001
try:
	ports = list(serial.tools.list_ports.comports())
	portIndex = [x[2] for x in ports].index("USB VID:PID=2341:0001 SNR=7523733353635130F171")
except:
	print "Arduino not found."
	print "Plug in Arduino."
	sys.exit()
	
SERIALPORT = "/dev/" + str(ports[portIndex][1])

# open serial port
try:
	ser = serial.Serial(SERIALPORT, 9600)
	ser.dsrdtr=False
	ser.setDTR(level=False) 
	# above line is needed to run code without Ardiuno IDE and Serial Monitor
except serial.SerialException:
	print "Failed to write to port %s." % SERIALPORT
	print "Plug in Arduino or connect to different port."
	sys.exit()
	
app = wx.App(0)
MyDialog(None, -1, 'cut_polish.py')
app.MainLoop()

Here is part of my Arduino code (I only included what I thought was important):

void getIncomingData()
{
  int i = 0;
  int j = 0;
  int flag = 1;
  while (flag == 1)
  {
    if(Serial.available() > 0) {    

      if(j < 4){
        inComingData[j] = Serial.read();
        //Serial.print(inComingData[j]);
        j++;
      } 

      if(i < inComingData[3] - 1 && j > 1) {  
        inComingFraction[i] = Serial.read();
        if(i == (inComingData[3]-2)){
          flag--;
         } 
         i++;
       }     
     }   
  }  
}
#2

I don’t understand how you could every access the same Arduino via the IDE serial monitor and your webapp at the same time. There is only one serial port, and it should only be possible for it to be opened by one process at a time.

You haven’t described the message format you’re using and I haven’t bothered to try to figure it out for myself from your code. If it was up to me, I’d send the length as an integer value in whatever units were required to achieve the resolution I needed - for example, thousandths of an inch. And if it was up to me, I’d use a textual encoding scheme with numbers send as ascii decimal values, comma separated if you need multiple values, with messages terminated by a linefeed.

From a quick glance at the Arduino code receiving data from the serial port I would not expect that to work because your code will read data when there is non available. There are plenty of examples showing how to handle serial port data - here’s one example:

// incomplete
void handleSerial()
{
    const int BUFF_SIZE = 32; // make it big enough to hold your longest command
    static char buffer[BUFF_SIZE+1]; // +1 allows space for the null terminator
    static int length = 0; // number of characters currently in the buffer

    if(Serial.available())
    {
        char c = Serial.read();
        if((c == '\r') || (c == '\n'))
        {
            // end-of-line received
            if(length > 0)
            {
                handleReceivedMessage(buffer);
            }
            length = 0;
        }
        else
        {
            if(length < BUFF_SIZE)
            {
                buffer[length++] = c; // append the received character to the array
                buffer[length] = 0; // append the null terminator
            }
            else
            {
                // buffer full - discard the received character
            }
        }
    }
}

void handleReceivedMessage(char *msg)
{
	if(strcmp(msg, "on") == 0)
	{
		// handle the command "on"
	}
	else
	{
		// etc
	}
}
#3

PeterH:
I don't understand how you could every access the same Arduino via the IDE serial monitor and your webapp at the same time. There is only one serial port, and it should only be possible for it to be opened by one process at a time.

I agree with you, that's why I think this is a problem. In the end, I don't want to use the serial monitor. So there should only be one program accessing the port at a time: the python program which sends the code and then closes its serial connection and then the Arduino program reading in the values. With the added delay, it works. With the serial monitor, it works. I don't see why either should work.

PeterH:
You haven't described the message format you're using and I haven't bothered to try to figure it out for myself from your code. If it was up to me, I'd send the length as an integer value in whatever units were required to achieve the resolution I needed - for example, thousandths of an inch. And if it was up to me, I'd use a textual encoding scheme with numbers send as ascii decimal values, comma separated if you need multiple values, with messages terminated by a linefeed.

I am sending ASCII using chr(i), which returns a string of one character whose ASCII code is the integer i. For example, chr(97) returns the string 'a'. The format I am sending/receiving does not change the problem I am having, so I didn't think that was relevant.

PeterH:
From a quick glance at the Arduino code receiving data from the serial port I would not expect that to work because your code will read data when there is non available. There are plenty of examples showing how to handle serial port data - here's one example:

I apologize, but I don't see how your code and my code differ. Both say "if something is available, add data to an array/buffer". I know in advance how much data I am sending/receiving, so I am not waiting for '\r' or '\n', and also, my array will never overflow.

#4

There is only one serial port, and it should only be possible for it to be opened by one process at a time.

The "only one application at a time" restriction is only imposed on Windows. On other OS's, any number of applications can access the same serial port.

#5

PaulS:
The "only one application at a time" restriction is only imposed on Windows. On other OS's, any number of applications can access the same serial port.

I am using Linux (crunchbang).

#6