perl: problems reading serial interface

Forum 2005-2010 (read only) Hardware Interfacing
1

I have a perl-script-subroutines which listen /dev/ttyUSB0 under ubuntu:

Perl:
#after the port initialization

sub read_in {
$port->write("Whatever you feel like sending");

Poll to see if any data is coming in

my $char = $port->lookfor();

If we get data, then print it

Send a number to the arduino

if ($char) {
print "Recieved character: $char \n";
}
}

On the Arduino is the graph-code:

void setup() {
// initialize the serial communication:
Serial.begin(9600);
}

void loop() {
// send the value of analog input 0:
Serial.println(analogRead(0));
// wait a bit for the analog-to-digital converter
// to stabilize after the last reading:
delay(10);
}

but the string which read the subroutines are often (1 of 5 times) not complete.

Someone has an idea for an arduino and perl code, which sends data only when the perlscript are reading?

2

How would the Arduino know if the perl script was listening?

The Arduino could send a "Hey, I'm ready" kind of message. If it got a reply, it would know. Right now, you don't send that message.

The Serial.println function sends bytes, ints, and characters. Your variable naming and print statement do not reflect this reality.

How do you know that "the string which read the subroutines are often (1 of 5 times) not complete." (whatever that means) is true?

The Arduino is sending. The perl script is reading. There's a wire in between. Where would the missing data (if anything is missing) go?

There is a possibility that you are overflowing the transmit buffer. The possibility that this is happening could be reduced by delaying longer after the analogRead/Serial.println statement.

3

The code that emmele uses is actually (almost precisely) copied from the Examples for Analog reading and Serial communication, so one would expect it to work.

void setup() {
Serial.begin(9600);
}

void loop() {
int analogValue = analogRead(0);
Serial.println(analogValue);
delay(10);
}

I (and probably) emmele expect, that this is just fine for the Arduino.

The Arduino IDE features a Serial Monitor. I've loaded this sketch to my Arduino and I've connected a potentiometer to it. I can see the value changing, as I turn the potentiometer, everything seems to work just fine.

Then I try reading the serial port with Perl and every now I then there is an incomplete value e.g.

344
344
343
342
42
2
342
341

It appears as if the Perl interface is somehow mis-reading the values. Or maybe I am entirely wrong and need to learn how serial communication works.

Any ideas on that? How does the Serial Monitor in the IDE work? That one manages to catch all the values correctly, maybe I could just do the same in Perl?

4

Well, to be honest - it's doing largely what you've asked it to do. To look for some character (which hasn't been specified, so it's presuming a default of "\n"), and return when that character is found-- but it's also non-blocking, so it will happily return nothing even though something may be coming up soon. It doesn't appear to be missing any data for you, but instead returning it at a different time than you expect.

To quote the perldoc: "The lookfor method is designed to be sampled periodically (polled). Any characters after the match pattern are saved for a subsequent lookfor. Internally, lookfor is implemented using the nonblocking input method when called with no parameter. If called with a count, lookfor calls $PortObj->read(count) which blocks until the read is Complete or a Timeout occurs. The blocking alternative should not be used unless a fault time has been defined using read_interval, read_const_time, and read_char_time."

So, you could do it in a loop:

 my $value;
 while( ! defined($value) || ! length($value) ) {
    $value = $serial->lookfor();
 }

(This can, of course, loop forever - see below...)

While the OP is on linux, and thereby using Device::SerialPort, I don't know if yak is using win32 or linux. I've had issues with the reliability of the built-in timeouts in the Win32 API, so I prefer the good-ol-obvious using an alarm within an eval block to do blocking reads, and timeout if you don't get back the data in a reasonable timeframe. Sadly, even the blocking form of read() is unreliable on win32 platforms.

I have no problems whatsoever with the following method on any platform (win32, linux, osx, etc.) that has a good serial port class.

I've modified this slightly to do the same as your 'lookfor()' with a newline as the target, and make it so that it gives up after 15 seconds if it didn't get a line... You can make it timeout at whatever you want..

(Note, not compiled, may have typos g)

my $string = '';

  eval {
      
        my $chr;

      local $SIG{'ALRM'} = sub { die("TimeOut"); };

            # expect a complete string within 15 seconds

      alarm( 15 );

            
        while( $chr ne "\n" ) {

            # we use a while loop, as the serial object may
            # return nothing if the buffer is empty for a 
            # period of time. wrap entire thing in an alarm
            # so we don't wait forever, and can capture a timeout
            
           while( ! defined($chr) || ! length($chr) ) {      
            $chr = $serial->read(1);
         }
          
           $string .= $chr;
         }
      
            # disable alarm
            
      alarm(0);
      
};  # end eval block

  # check for error
if( $@ ) {
   if( $@ =~ /TimeOut/ ) {
      return($string);
   }
   else {
      die($@);
   }
}

!c

5

Hey, thank you for the input. It actually made me look into the stuff and I think I have my solution. What I have done is this:

Arduino: Wait until someone asks us to send data. Upon request, send an analog reading.

Perl: In the main loop: Send a data request to the Arduino. Perform a data reading afterwards.

Arduino

int myval;
int sensorValue;

void setup() {
  Serial.begin(9600);
}

void loop() {
  
  if (Serial.available() > 0) {
    myval = Serial.read();
    
    if (myval) {
      sensorValue = analogRead(0);
      Serial.println(sensorValue);
    }       
  }
}

Perl

#!/usr/bin/perl
use Device::SerialPort;

my $file = "/dev/ttyUSB0";

my $port = Device::SerialPort -> new($file);
$port -> baudrate(9600);
$port->write_settings();


open(DEV, "<$file") or die; 

while (1)
{
      # perform whatever needs to be done in this loop
      
      $port->write(1); # send something to the Arduino

      if ($_ = <DEV>) { print $_, "\n"; } # read serial port and print results
}

I'm sure it has many drawbacks and possible failures so any input is welcome.

6

How fast is your application?

I changed my code to make it faster: I send all 6 analog signals in one string. And I read with my perl-script a string with double length. With regular expressions I cut the correct 6 analog-datas out from the string.

But I lost anyway 60% of the data.

Arduino-Code

void setup() {

  Serial.begin(115200);

}

void loop() {

  char signal_t[29];
  sprintf(signal_t,"b %04d %04d %04d %04d %04d %04d e",analogRead(0),analogRead(1),analogRead(2),analogRead(3),analogRead(4),analogRead(5));
  Serial.println(signal_t);

  delay(2);


}

Perl-Snippet:

my $schnittstelle = Device::SerialPort->new($serialport);
         if($schnittstelle){
         $schnittstelle->stopbits($stopbits);
         $schnittstelle->baudrate($baudrate);
         $schnittstelle->databits($databits);
         $schnittstelle->parity($parity);
         $schnittstelle->write("$write");
         $char =  $schnittstelle->read(60);
         if ($char) {
             if ($char =~ /b\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+e/){
                  return(1,$1,$2,$3,$4,$5,$6,$par_3);

             }  else {
            
             }
           } 
           }
7

emmele,

Why are you reading 60 bytes, and then throwing away 30?

char signal_t[29];
...
Serial.println(signal_t);

$char =  $schnittstelle->read(60);
         if ($char) {
             if ($char =~ /b\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+e/){
                  return(1,$1,$2,$3,$4,$5,$6,$par_3);

You're waiting to read two sets values, but then attempting to parse only one, and throwing the other away?

Why not just try something like this for a minute, and see how it works for you:

use Time::Hires;

 # setup your serial stuff here...

while( 1 ) {
   my $chr = $schnittstelle->read(1);
   if( defined($chr) && length($chr) ) {
     print $chr;
   }
   sleep(0.01);
}

.. and just see if you get all of your bytes =)

Yak: FWIW, be careful with

if($_ = <DEV>) {
  ..
}

It's not a good practice to assign magic variables yourself.

while(<DEV>) {
  print;
}

Also note - be careful there, you need to make sure your input record separator is set correctly (by default it's "\n") when slurping lines from a file handle like that. A lot of people get burned that way.

!c

8

I read 60 characters because so I'm sure that there is a full data-set inside. Is not a clean way. I now this.

With:

if( defined($chr) && length($chr) ) {
     print $chr;
   }

often I need more then one try to take a correct dataset.
I tested both, and for my application (plotting data from all 6 analog-input with perl/tk). I received more data with the 60char-method.

I tried something like a handshake, but I'm not expert in serial connecting and so I'm not able to get it work.

Where I do:

if($_ = <DEV>) {

? You mean this code:

if ($char =~ /b\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+e/){
                  return(1,$1,$2,$3,$4,$5,$6,$par_3);
             }
9

Emmele,

If you notice, my code had a while(1) loop wrapped around it. There should be no data loss, because it should print every single character received over the serial, as received, for as long as the script runs. The reason I wrote that was to show you that you're not losing data (that would be highly suspicious =), but that the read(60) is not necessarily doing what you think it is =)

read(60), lookfor(), etc. all have "magic" happening under the hood. That is -- there are several parameters controlling their behaviors. My point in showing you that body of code was that you can avoid all of that "magic" and do it without assistance.

Hence:

while(1) {
  my $chr = $serial->read(1);
  print $chr;
}

Will never lose any characters from the serial. The test I did in there:

if( defined($chr) && length($chr) ) {
     print $chr;
   }

Goes to demonstrate the point that read(1) doesn't always return valid data. It will sometimes return defined-but-empty.

So, let me clarify -- if you run this script exactly as I type it below, and lose data, I'll be a monkey's uncle:

use Device::SerialPort;
use Time::HiRes;

my $serialport = '/dev/whatever';
my $write = "whateveryouwrite";

my $schnittstelle = Device::SerialPort->new($serialport);
if($schnittstelle){
   $schnittstelle->stopbits($stopbits);
   $schnittstelle->baudrate($baudrate);
   $schnittstelle->databits($databits);
   $schnittstelle->parity($parity);
   $schnittstelle->write("$write");

   while(1) {
      my $chr = $serial->read(1);
      if( defined($chr) && length($chr) ) {
         print $chr;
      }
   }
 }

Don't be fancy, don't try to add it in to your existing stuff. Just run that, see that it works, and then figure out how to get fancy :slight_smile:

Basically, I'm telling you your problem is not that dat is being lost, but that you're neither using read() nor lookfor() correctly. There is no guarantee that they will do explicitly what you state. You have to validate!

The comment about

if($_ = <DEV>) {

Was directed at Yak, not you :slight_smile:

!c

10

Thanks,

I will try it next sunday. I'm outside for a week

bye