Tty for serial port to Arduino from Linino

Very well done wayoda. This is become really interesting. Out of curiosity: could you set up a stress test case?
Something like flooding the 32u4 with random chars (in a 1.000.000 rounds "for" loop) and checking if the output is the expected one

Great job!!! This is what I was looking for.
I'll try.

I did some quick tests. In short...

  • The maximum baudrate is 115200 baud. Tests with 230400 baud always failed on the first read attempt.
  • With the simple Arduino Sketch (see below) the IO-throughput is about 10kB/s.

The baudrate in the python script and the Arduino Sketch must (obviously) match to make this work

Here is the python script for testing

"""
Usage :
    python st.py PORT BAUDRATE KILO_BYTES

example :
python st.py /dev/ttyATH0 115200 10
Will send and receive 10kB of data

turn on profiling:
python -m cProfile st.py /dev/ttyATH0 115200 10
"""

import serial
import sys

data='01234567012345670123456701234567'
expected='12345678123456781234567812345678'

#one packet is 32 bytes so 1kByte is 32 packets
packets=int(sys.argv[3])*32
s=serial.Serial(sys.argv[1],sys.argv[2],timeout=5)
for i in range(packets):
    s.write(data)
    result=s.read(32)
    if result!=expected:
        raise ValueError('Mismatch on run : '+str(i)+'\ndata = "'+data+'"\nresult = "'+result+'"')
print "Done!"
s.close()

On the Arduino I run this Sketch. It blinks the Led after coming out of a reset.

void setup() {
  pinMode(13,OUTPUT);
  digitalWrite(13,HIGH);
  delay(500);
  digitalWrite(13,LOW);
  delay(500);
  digitalWrite(13,HIGH);
  delay(500);
  digitalWrite(13,LOW);
  Serial1.begin(115200);
  
}

void loop() {
  char c;
  
  while (Serial1.available() > 0) {
    c=Serial1.read();
    c++;
    Serial1.print(c);
  }
  
}

Here is the output of the longest test I ran (4MB of data).
Takes about 7 minutes to complete!

root@YunYun:/mnt/sda1/python# python -m cProfile st.py /dev/ttyATH0 115200 4096
Done!
         5392556 function calls in 431.015 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.005    0.005    0.026    0.026 __init__.py:8(<module>)
        1    0.017    0.017    0.022    0.022 serialposix.py:13(<module>)
        1    0.000    0.000    0.000    0.000 serialposix.py:163(Serial)
        1    0.000    0.000    0.001    0.001 serialposix.py:168(open)
        1    0.001    0.001    0.001    0.001 serialposix.py:192(_reconfigurePort)
        1    0.000    0.000    0.000    0.000 serialposix.py:321(close)
   131072  189.347    0.001  392.045    0.003 serialposix.py:340(read)
   131072   11.837    0.000   25.662    0.000 serialposix.py:359(write)
        1    0.001    0.001    0.001    0.001 serialutil.py:111(SerialBase)
        1    0.001    0.001    0.002    0.002 serialutil.py:123(__init__)
        1    0.000    0.000    0.000    0.000 serialutil.py:196(setPort)
        1    0.000    0.000    0.000    0.000 serialutil.py:221(setBaudrate)
        1    0.000    0.000    0.000    0.000 serialutil.py:240(setByteSize)
        1    0.000    0.000    0.000    0.000 serialutil.py:253(setParity)
        1    0.000    0.000    0.000    0.000 serialutil.py:266(setStopbits)
        1    0.000    0.000    0.000    0.000 serialutil.py:279(setTimeout)
        1    0.000    0.000    0.000    0.000 serialutil.py:298(setWriteTimeout)
        1    0.000    0.000    0.000    0.000 serialutil.py:30(SerialException)
        1    0.000    0.000    0.000    0.000 serialutil.py:317(setXonXoff)
        1    0.000    0.000    0.000    0.000 serialutil.py:328(setRtsCts)
        1    0.000    0.000    0.000    0.000 serialutil.py:339(setDsrDtr)
        1    0.000    0.000    0.000    0.000 serialutil.py:35(SerialTimeoutException)
        1    0.000    0.000    0.000    0.000 serialutil.py:355(setInterCharTimeout)
        1    0.000    0.000    0.000    0.000 serialutil.py:40(FileLike)
        1    0.003    0.003    0.004    0.004 serialutil.py:8(<module>)
        1   13.242   13.242  431.015  431.015 st.py:14(<module>)
        3    0.000    0.000    0.000    0.000 {_struct.pack}
        2    0.000    0.000    0.000    0.000 {chr}
        3    0.000    0.000    0.000    0.000 {getattr}
       21    0.000    0.000    0.000    0.000 {hasattr}
   131072    1.061    0.000    1.061    0.000 {isinstance}
  2565176   21.640    0.000   21.640    0.000 {len}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {posix.close}
        1    0.000    0.000    0.000    0.000 {posix.open}
  1151516   62.691    0.000   62.691    0.000 {posix.read}
   131072   12.070    0.000   12.070    0.000 {posix.write}
        1    0.038    0.038    0.038    0.038 {range}
  1151516  119.062    0.000  119.062    0.000 {select.select}
        1    0.000    0.000    0.000    0.000 {termios.tcgetattr}
        1    0.000    0.000    0.000    0.000 {termios.tcsetattr}

Can you retry with 250000 baud? That's the speed we set for ttyATH0

Doesn't work. For some reason I get incomplete readings for all speeds above 115200 baud. (230400, 250000 etc.)
I also tried to write smaller data packets of 16 bytes instead of 32 bytes, but no luck. Every baudrate above 115200 baud fails on the first write/read cycle.

But 115200 baud (10kB/s) is fast enough for most usecases where data from the ATmega32u4 is being forwarded to a webservice.
I can live with that limitation.

I got Firmata to work between a Python script running on Linino and the Arduino. I was able to get about 400 messages per second which is sufficient for my needs. I made the following changes to the StandardFirmata sketch "begin" section.

void setup() 
{
  
  Firmata.setFirmwareVersion(FIRMATA_MAJOR_VERSION, FIRMATA_MINOR_VERSION);

  Firmata.attach(ANALOG_MESSAGE, analogWriteCallback);
  Firmata.attach(DIGITAL_MESSAGE, digitalWriteCallback);
  Firmata.attach(REPORT_ANALOG, reportAnalogCallback);
  Firmata.attach(REPORT_DIGITAL, reportDigitalCallback);
  Firmata.attach(SET_PIN_MODE, setPinModeCallback);
  Firmata.attach(START_SYSEX, sysexCallback);
  Firmata.attach(SYSTEM_RESET, systemResetCallback);

  Serial1.begin(9600); // Set the baud.
  Firmata.begin(Serial1);
  systemResetCallback();  // reset to default config
}

On the python I installed PyFirmata and pyserial. The following code turns on the LED:

from pyfirmata import Arduino, util
board = Arduino('/dev/ttyATH0', baudrate=9600)
board.digital[13].write(1)

Hi Wayoda,

when you say a BAUD rate of 250000 fails - do you get characters from Serial, or nothing at all? The Arduino UART should definitely do 250000 , and higher. And since the hard-wired serial connection is very short, there shouldn't be any signal degradation issues.

Maybe you have a marginal board. I will give it a go tonight and let you know results. If necessary we can look at the UART registers for over-run or parity errors....

Con

GreyCon:
Hi Wayoda,
when you say a BAUD rate of 250000 fails - do you get characters from Serial, or nothing at all?

The python script writes 32 chars to the ATmega and then reads 32 bytes with a five second timeout.
When used with baudrates above 115200 the read returns unexpected characters and also too few of them.

GreyCon:
The Arduino UART should definitely do 250000 , and higher. And since the hard-wired serial connection is very short, there shouldn't be any signal degradation issues.
Maybe you have a marginal board. I will give it a go tonight and let you know results. If necessary we can look at the UART registers for over-run or parity errors....

I tried everything to create a bad environment for the Yun to provoke an error at 115200 baud...

  • Wind an active Lan-cable several times round the Yun
  • Place the Yun on to of my cellphone, and call myself
    ...but the python script runs without problems

Raise the speed to 250000 and it fails .

BTW:
Sketch programming from the Yun side (not over USB) does not use the serial port. It uses the avrdude -c linuxgpio option which emulates a SPI programmer. See file /etc/avrdude.conf line 976

Hi,
there is a serious problem with my little echo sketch on the ATmega.

The changes in /etc/inittab detach the serial line to the ATmega, but this happens very late in the boot process.
Until just before the login prompt for the shell is shown, all kernel messages are still forwarded to the ATmega.
Because my test sketch more or less echos what is read from the serial, the boot process seems to be stopped by the random data coming back over Serial1.
The effect is that because of the echo sketch the WiFi setup never completes. Login over ssh is not possible.

When the simple Blink Sketch or even the Yún Serial Terminal runs on the ATmega the Yun boots fine because these Sketches don't throw any random data back at the linux machine. If you are stuck like that because you tried the test at home, simply upload a Sketch not writing to Serial1. http://arduino.cc/en/Tutorial/YunSerialTerminal

For an application using the Serial1 to communicate with the ATMega this means, you are only allowed to listen on the Serial1 until the Linux machine was sucessfully booted.

If you look in the Bridge.cpp source code you'll se that they deal with this.

 // Wait for U-boot to finish startup
  do {
    dropAll();
    delay(1000);
  } while (stream.available()>0);

This doesn't seem very fool proof. Its counting on the fact that during the boot process you see some data at least once per second come over serial. If there was a > 1 sec pause then you'd be out of luck.

I'd be happy to see a robust code snippet I could insert into my sketches.

Good catch.

Hmmm....I have used the serial console of the early boot process before to debug a bad dd-wrt (openwrt fork) image on a router. I suspect that this "feature" is enabled by default in the openwrt build process since this is a relatively good way to unbrick a board with a corrupt flash image.

A bootlog for linino is at: linino/bootlog at master · arduino/linino · GitHub. There are several opportunities that echoing the console is not a good idea!

Took a bit of digging, but it appears the only way to stop the "terminal chatter" early in the boot process is to rebuild the kernel to disable "Early printk". Not something that most folks (including me) will be excited about for many reasons. Seems like the best workaround is exactly what scrot and wayoda are doing: find a way to make sure the 32u4 does not respond to this "terminal chatter".

Indeed, disabling printk doesn't seem a very good idea: I fear we may lose dmesg output, making device debugging (wifi in particular) impossible

Just a warning to the wise, it looks like this breaks the WiFi reset button, re-enabling
ttyATH0::askfirst:/bin/ash --login in /etc/inittab makes it work again.

If you lose your Yun this way check out my post about "is my WiFi dead?"

Hi,
I had a little accident in my workshop, so I will not be able to do any serious work with my yun, but I took some time to read the whole documentation that is available, and there might be a nice way to signal the atmega when the linux machine is available. The original Yun docs mention a heartbeat line from the mips-processor (GPIO 19) to the arduino Pin 7.

I should be possible to write a Linux Init-script that switches this handshake to a certain state when the Linux part is fully booted and the Console is free to use for an application. The same is possible for situations where the Linux part is shutting down or in a Reboot.

An Arduino Sketch would simply have to monitor Pin 7 and thereby knows when serial IO with som Python code is possible.
One-handed typing is a bit arkward, so here are just the references:

in both document search for "handshake"

While you are investigating on your own, do you mind opening on issue on GitHub - arduino/YunBridge ? This is pretty cool stuff and it would be a shame to forgot it just because we are stuck with other tasks at the moment

It works for me... thank you. I've working on that and at the end you gives the solution.

scrot:
I got Firmata to work between a Python script running on Linino and the Arduino. I was able to get about 400 messages per second which is sufficient for my needs. I made the following changes to the StandardFirmata sketch "begin" section.

void setup() 

{
 
  Firmata.setFirmwareVersion(FIRMATA_MAJOR_VERSION, FIRMATA_MINOR_VERSION);

Firmata.attach(ANALOG_MESSAGE, analogWriteCallback);
  Firmata.attach(DIGITAL_MESSAGE, digitalWriteCallback);
  Firmata.attach(REPORT_ANALOG, reportAnalogCallback);
  Firmata.attach(REPORT_DIGITAL, reportDigitalCallback);
  Firmata.attach(SET_PIN_MODE, setPinModeCallback);
  Firmata.attach(START_SYSEX, sysexCallback);
  Firmata.attach(SYSTEM_RESET, systemResetCallback);

Serial1.begin(9600); // Set the baud.
  Firmata.begin(Serial1);
  systemResetCallback();  // reset to default config
}




On the python I installed PyFirmata and pyserial. The following code turns on the LED:



from pyfirmata import Arduino, util
board = Arduino('/dev/ttyATH0', baudrate=9600)
board.digital[13].write(1)

Thanks for the update. Here's my Firmata setup(). Its very similar to yours. The only difference is a section to wait for the U-boot to complete, and I set the baud to 115200 which works like a champ. I stole the U-boot section from Bridge.cpp.

void setup() 
{
  Serial1.begin(115200); // Set the baud.
  
   // Wait for U-boot to finish startup.  Consume all bytes until we are done.
  do {
     while (Serial1.available() > 0) {
        Serial1.read();
        }
    
    delay(1000);
  } while (Serial1.available()>0);
  
  Serial.begin(9600); // For logging.
  
  Firmata.setFirmwareVersion(FIRMATA_MAJOR_VERSION, FIRMATA_MINOR_VERSION);

  Firmata.attach(ANALOG_MESSAGE, analogWriteCallback);
  Firmata.attach(DIGITAL_MESSAGE, digitalWriteCallback);
  Firmata.attach(REPORT_ANALOG, reportAnalogCallback);
  Firmata.attach(REPORT_DIGITAL, reportDigitalCallback);
  Firmata.attach(SET_PIN_MODE, setPinModeCallback);
  Firmata.attach(START_SYSEX, sysexCallback);
  Firmata.attach(SYSTEM_RESET, systemResetCallback);

  Firmata.begin(Serial1);
  systemResetCallback();  // reset to default config
}

Hi,
I use the Linino to handle REST Web Services, and JSON data. These services talk to an API layer, which maps the JSON data to "device level" calls, which my Arduino understands. (Eg replace Timers for Monday, read all timers, etc.)

This API layer takes a sequence of bytes (A private command or the Arduino API), wraps it in a frame (SOM BYTE, frame datalen, DATA, EOM, and Checksum) and sends it to the Arduino over the serial port.

I have a C++ class (YUNListener) on the Arduino which waits for SOM, parses the data, ensures there's an EOM, and validates the Checksum. So ant comms errors, or other unwanted / unexpected incoming data on the serial bridge are discarded. The C++ class has 2 Virtual methods, which do nothing in the base class, but are designed to be over-ridden in real subclasses. (Eg TimerListener : YUNListener)

The methods are:

  1. handleCommand( data ) which you can implement, and will be given a correct byte array from incoming commands.
  2. debug( data ) which will be called at various points in the command parsing process. I use this method to check for an input pin being pulled high, and if it is, I output serial debugging info to an output pin . So I have a cable that connects the output pin to a serial terminal, and straps the other pin high. So I automatically get debug output when I connect the cable to my laptop.

Happy to supply the code if anyone is interested.

Please show us the code, or post a link to GitHub. I'd be very interested in seeing your code. Its probably simpler than Firmata which would be great.