Connection using pyserial doesn't work properly

Hiho,

i have a project using an Arduino Duemilanove to communicate with a Python-skript on my ubuntu-11.10-machine. The Skript is running as a deamon, that starts when the computer is coming up and the Ardunino-Board is detected on something like ttyUSB0. Now the skript waits for an input from the Arduino and process it.
Python 2.7
pyserial 2.6

Well, the arduino-skript works fine, and the communication with the Arduino-IDE (1.0) aswell. But with the connection that the python-skript tries to establish is something very odd.
After a long time testing I had very chaotic results of sometimes working, sometimes not or with corrupted input from the Board.
At the moment I can observe the following behavior:
When the computer is coming up and detects the arduino, the skript starts and establish a connection.
The Board restarts.
But I get no input.
When ever I restart the deamon (connection closed and re-established), the board restarts aswell.
But no proper communication.
Now I stop the deamon and start the Arduino-IDE.
The board restarts and everything works fine.
I stop the IDE and restart the deamon.
The Board restarts and everything works fine!!!
I restart the deamon, but the board does NOT restart anymore. Even so the connection is still working.

I tried a lot of stuff, like flushing the connection-input (port.flushInput()) right after opening it. Or try to set port.setRTS(False) (but I'm not sure what it means). I tried a lot of delay-time-stuff to synchronize the starting-process of the board and the deamon. Most time I've got some input, but either only the line-ending of a word or only few characters like "pwr_f" of "power_off". At the moment I get nothing or everything (like described above) but cannot say what excactly makes the difference.
I also tried to reload the ftdi_sio- and usbserial-driver just before establishing the connection. But it didn't helped.

Well, here is some code. It would be gorgeous if somebody could help me... I really don't know any further.

I tried to cut the code to the essentials...

// outputs
int pin_led_red = 12;
int pin_led_green = 13;
int reply = 0;
// timer
unsigned long timer;
long fifty_sec = 50000L;
long half_sec = 500L;

void setup() {
  delay(1500);
  Serial.begin(9600);
  // set pin-modes
  pinMode(pin_led_red, OUTPUT);
  pinMode(pin_led_green, OUTPUT);
  // init-signal
  blink(pin_led_green, 500, 100);
  // test serial connection
  test_connection();
}
void loop() {
  // some other code-stuff
}
// test the serial connection
void test_connection() {
  Serial.println("check");
/*  Serial.flush();*/
  reply = wait_for_reply(fifty_sec);
  if (reply == 520) {
    blink(pin_led_green, 3000, 300);
  } else {
    blink(pin_led_red, 3000, 300);
  }
}
// wait for an answer and return it as crossfoot
int wait_for_reply(long wait) {
  timer = millis();
  while (millis() - timer < wait) {
    reply = Read();
    if (reply) {
      Serial.println(reply);
      break;
    }
    delay(20);
  }
  return reply;
}
// get coming-in-data and return as crossfoot
int Read() {
  reply = 0;
  while (Serial.available() > 0) {
    reply += Serial.read();
    delay(10);
  }
  return reply;
}
// let a led blink for 4 seconds
void blink(int led, int duration, int speed) {
  timer = millis();
  while (millis() - timer < duration) {
    digitalWrite(led, HIGH);
    delay(speed);
    digitalWrite(led, LOW);
    delay(speed);
  }
}
#!/usr/bin/python

import serial, time, logging, os, subprocess, sys, traceback, signal
from serial.tools import list_ports
from fotobox.mailing import Email

#initiate logging
LOG_FILE = "/var/log/arduino_control.log"
logger = logging.getLogger('')
logger.setLevel(logging.DEBUG)
f = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh = logging.FileHandler(LOG_FILE)
fh.setFormatter(f)
logger.addHandler(fh)


class ArduinoControl:
	def __init__(self, dev):
		signal.signal(signal.SIGTERM, self.signal_handler)
		self.dev = dev
		self.run = True
		self.open_port(dev)
		self.go()

	def open_port(self, dev):
		self.port = serial.Serial(dev, 9600)
		logger.debug(self.port.getSettingsDict())

	def process_data(self, data):
		if data == 'check':
			self.port.write(data + '\n')
			logger.info('written: "%s"', data)

	def go(self):
		while self.run:
			try: data = self.port.readline().strip('\r\n')
			except: self.error_handling()
			if not data: continue
			logger.info('received: "%s"', data)
			self.process_data(data)
			time.sleep(0.01)

	def error_handling(self):
		lines = traceback.format_exception(*sys.exc_info())
		lines = [l[:-1] for l in lines]
		logger.error(' ### '.join(lines))

	def signal_handler(self, num, frame):
		self.run = False
		self.port.flushInput()
		self.port.flushOutput()
		self.port.close()
		logger.info(num)
		sys.exit()

def main():
	port_name = sys.argv[1]
	logger.debug(port_name)
	arduino = ArduinoControl(port_name)


if __name__ == "__main__": main()

I wonder if you should try some sort of ack/nack/timeout protocol like when the arduino wakes up it sends a syn character and waits for an ack in return. If it doesn't get one after (say) 500 ms it tries again. Once it gets an acknowledgement hack you can go to whatever protocol you had planned. Your python script could reset the arduino when it starts and, if it doesn't see a syn within some time, reset it agaIn.

There's probably some mOre robust variant of this but you get the idea.

Thank you for your reply.
Well. Actually I do something like this:
In setup() I call test_connection(). It sends a "check", and then waits for a reply (wait_for_reply(fifty_sec)). If it got one (and it is the right one) the green led goes blink-blink-blink and real life can start.
Ok, the python-skript do not restart the board if nothing is coming. (Would you do that by closing the port and open it again?). But this cannot be the solution. Because I need this stuff to work 99% reliably. And however, this Arduino-IDE-program nearly always works. So there must be a systematical problem with the way my skript establish the connection. Really, I have no idea what I did wrong. But I want to get this thing run.
I tested it again, and the fact is, that as soon as I once have let the Arduino-IDE connected to the board, it will work with my program as well. So there must be a setting on system-level, that once set from the Arduino-IDE is still in effect when my program reconnect to the board again. That is what I think. But I have no idea what it could be, and how I could apply it myself...

I can't speak to the linux serial open differences but this Arduino Playground - Python does sPeak to them.

Thanks for the tip.
I've already red that side. I now tried also the arduino_serial.py-skript that is linked at the bottom, but I've got the same problem...

ok. Feels like kind of a dirty hack, but finally I've got it working...

I played around with the parity-option of the python-serial-modul, and it appears that if I choose serial.PARITY_ODD I've got some communication, even if the letters where wrong and weird. But after switching back to the default-value (serial.PARITY_NONE) everything works fine. Even though I don't no why that is so, I put something in my code like

self.port.parity = serial.PARITY_ODD
self.port.open()
self.port.close()
self.port.parity = serial.PARITY_NONE
self.port.open()

and the fact is, that it didn't failed one time until now. So I'm confidently, that it will help me along for the next time...
If somebody has a good idea how it comes that it works this way, then please let me participate in it...

best wishes...