Arduino Yun - Python script crashing when called from Arduino

Official Hardware Classic Yún
1

I have an issue with a .py script crashing when it's called from the Arduino side of the Yun. The strange thing is that the .py script does NOT crash when I call it from PuTTY, from the shell. I can clearly see on the serial monitor when the process stops running because process.running() outputs 0 when the .py script crashes.

My goal is to have the python script start any time it's find to be not running. I have tried to run it at startup through etc/rc.local, but that's not working. My concern with using rc.local is also that if the script dies at some point after startup, there's no way to start it again without a reboot.

Eventually I will have no physical access to this system, so I'd like this process to be automatic.

From the serial monitor, I can see when 'relay' is set to 9, which happens every minute at the 30 second mark. However at 0, 20 or 40 minutes, the script crashes (process.running()=0). This does not happen when the script is called from PuTTY.

Any help would be appreciated, thanks!

Here is my current python code:

from ftplib import FTP
import datetime
import sys

sys.path.insert(0, '/usr/lib/python2.7/bridge')
from bridgeclient import BridgeClient as bridgeclient
from time import sleep

# Initialize vars
ftpIP = '[redacted]'  # Internal IP
# ftpIP = '[redacted]'      #External IP
ftpDirectory = '3990548'
value = bridgeclient()
value.put('relay', '0')
status = 'Disarmed'



# Initialize FTP
ftp = FTP(ftpIP)
ftp.login('[redacted]', '[redacted]')
ftp.cwd(ftpDirectory)




# Loop
while (1):

    #Don't eat all CPU cycles
    sleep(0.01)

    # Define minute and second, each loop iteration
    rawTime =   datetime.datetime.now()
    fmtTime =   rawTime.strftime("%m/%d/%Y %H:%M:%S")
    date =      datetime.date.today()
    hour =      rawTime.hour
    minute =    rawTime.minute
    second =    rawTime.second
    month =     date.month
    day =       date.day
    year =      date.year

    fileDate =  rawTime.strftime("%m%d%y")
    fileName =  '{}.txt'.format(fileDate)
    filePath =  'tadtest/{}'.format(fileName)


    # Keep FTP alive
    if (second == 30):
        ftp.voidcmd('NOOP')

        value.put('relay', '9')
        print value.get('relay')
        sleep(1)
        value.put('relay', '0')

    ### Alarm System ###
    if (second < 6 and (minute == 0 or minute == 20 or minute == 40)):

        with open(filePath, "a+") as myFile:
            myFile.write('SensorAlarm, {}\n'.format(fmtTime))

        if (status == 'Disarmed'):
            # Trigger Keyfob Relay Arm (3)
            value.put('relay', '3')
            print value.get('relay')
            sleep(6)
            value.put('relay', '0')

            print "Arm"
            status = 'Armed'

        elif (status == 'Armed'):
            # Trigger Contact Sensor Relay
            value.put('relay', '2')
            print value.get('relay')
            sleep(6)
            value.put('relay', '0')

            print "Alarm"
            status = 'Armed'

        else:
            print "Error in Alarm (Arm) 'if' statement"

        # Write to FTP
        ftp.storbinary('STOR {}'.format(fileName), open('/mnt/sda1/{}'.format(filePath), 'rb'))

Arduino code:

#include <Bridge.h>
#include <Adafruit_GFX.h>
#include <Adafruit_ST7735.h>

//Define LCD variables
#define TFT_CS    7
#define TFT_RST   8
#define TFT_DC    9
#define TFT_SCLK 10
#define TFT_MOSI 11
Adafruit_ST7735 tft = Adafruit_ST7735(TFT_CS, TFT_DC, TFT_MOSI, TFT_SCLK, TFT_RST);
int crsr=0;
uint16_t mcolor=ST7735_RED;

Process p;

//Here we hold values coming from the bridge
char relayValue[2] = {0};

int previousRelayInt = 100;


void setup() {
  Bridge.begin();
  Serial.begin(9600);
  p.runShellCommandAsynchronously("python /mnt/sda1/ftp_test.py");
  
  //Relay pins
  pinMode(2, OUTPUT);
  pinMode(3, OUTPUT);
  pinMode(4, OUTPUT);
  pinMode(13, OUTPUT);

  //Zero out memory used for relay value
  memset(relayValue, 0, 2);

  //Initialize pins 
  digitalWrite(2, HIGH);
  digitalWrite(3, LOW);
  digitalWrite(4, LOW);
  digitalWrite(13, LOW);

  //Initialize screen to be black
  tft.initR(INITR_BLACKTAB);
  tft.fillScreen(ST7735_BLACK); 
}




void loop() {

  Serial.println(p.running());

//  if (p.running()==0)
//  {
//  p.runShellCommandAsynchronously("python /mnt/sda1/ftp_test.py");
//  delay(3000);
//  Serial.println("Started .py script again");
//  }
    
  Bridge.get("relay", relayValue, 2);
  int relayInt = atoi(relayValue);


  if (relayInt==previousRelayInt)
  {
  }
  else if (relayInt!=previousRelayInt)
  {
    printTFT(relayValue);
    Serial.println(relayValue);
    previousRelayInt = relayInt;
  }






  if (relayInt == 0) 
  // Do nothing
  { 
  }
  
  else if (relayInt == 2) 
  //Trigger Contact Sensor - Alarm
  {
    digitalWrite(2, LOW);
    printTFT(relayValue);
    delay(4000);
    digitalWrite(2, HIGH);
    delay(4000);

    mediaAlarm();    
  }
  
  else if (relayInt == 3) 
  //Trigger Keyfob - Arm
  {
    digitalWrite(3, HIGH);
    printTFT(relayValue);
    delay(4000);
    digitalWrite(3, LOW);
    delay(4000);

    mediaArm();
  }
  
  else if (relayInt == 4) 
  //Trigger Keyfob - Disarm
  {
    digitalWrite(4, HIGH);
    printTFT(relayValue);
    delay(4000);
    digitalWrite(4, LOW);
    delay(4000);

    mediaDisarm;
  }
  
  else if (relayInt == 9) 
  //Blink LED
  {
    digitalWrite(13, HIGH);
    printTFT(relayValue);
    delay(4000);
    digitalWrite(13, LOW);
    delay(1000);
  }        
}
2

Make python into 2 small python, and run them at cron.

0, 20 or 40 minutes:

*/20 * * * *  /root/p20.py

every 1 minute:

*/1 * * * *   /root/p1.py

cron job sample

3

tduff:
My goal is to have the python script start any time it's find to be not running.

That's a good idea, I do it myself. But I do it slightly differently than you (more on that later.)

I have not studied your code in detail, but I do see something in the Python code that jumps out at me:

   filePath =  'tadtest/{}'.format(fileName)

You are building a file path, and then you are trying to open that file - in the middle of the code that is running at the 0/20/40 minute time frame where the crashes occur. I believe your issue is that you are using a relative path to the file.

When you are running the script manually from PuTTY, I assume you have first set the current directory to be the directory where the script resides. And then I guess the tadtest folder is in that current directory? That would be why the script works when you run it manually.

But when you are running it from rc.local, or from a Process object in your sketch, you will not have the same current directory. That directory does not have your tadtest folder, so it cannot open your file, and it crashes.

Unless your script has explicitly set the current directory, it should always use fully qualified path names (they should always start with a slash character so the path starts at root.) Even if you do explicitly set the current directory in your script, it's not a bad idea to use a fully qualified path name. (I define the working directory as a "constant" string at the beginning of my Python code, so I only have to update it in one place if I change the directory name or location.)

Make it a fully qualified path name, I bet that will solve your crashes.

And test it by switching to a different directory, and then manually running it using the full path name of the script file. If it doesn't work that way, don't even bother trying to run it from rc.local, cron, or a Process object.

Not the cause of your crash, but a minor observation:

 if (p.running()==0)

p.running() returns a boolean - that is defined as zero is false, and non-zero is true. I don't like explicitly comparing a boolean to a number. It's already a logical value, treat it as one. I find this format to be easier to read and understand:

 if (!p.running())

It reads more naturally as "if not running" and is closer to what you are trying to do. Comparing it to zero is not as obvious, and it might lead you to try and compare it to 1 sometime in the future. That's where the problem comes in - when returning a true boolean value, the language only states that the value will be non-zero, not necessarily one. If you keep using numeric comparisons, you might be tempted to test against 1, and it may not work in all cases. The same thing happens when you are comparing equality (or non-equality) to true (which is usually defined as 1.) For example, you might be tempted to write:

 if (p.running() != 1)

That would work if p.running() always returns 1 if it's running, but will fail if it ever returns something else like 2 to signal running. That would be valid according to the boolean definition, but that code would not handle it properly.

Boolean values are logical values - use logical operators with them, not numerical operators. Using addition for OR, multiplication for AND, and == instead of NOT will work most of the time, but not always. it's better to use the explicit logical operators of &&, ||, and !.

Now, I mentioned I did the process restarting slightly differently. The issue is that you have the command that is being run hard coded in two different places. If you ever need to update the command string, you will need to update it in two places. The simple solution is to define a constant string with the command string, but that's still not an ideal situation - there may be additional statements required to launch the command, for example some .addParameter calls().

I create a function that checks and starts the process, something like this:

// Function to start or restart the Linux process.
// If the process is not currently running, it is started.
// Returns true if the process was just started, or false if it was already running.
boolean runProcess()
{
  boolean started = false;

  if (!p.running())
  {
    p.begin("python");
    p.addParameter("/mnt/sda1/ftp_test.py");
    p.runAsynchronously();
    Serial.println("Started .py script again");
    started = true;
  }

  return started;
}

Then, the function can be called in setup(), and in loop(). If it's necessary to know whether the process was just started (maybe you have to re-initialize something in your sketch to maintain synchronization) you can tell by the return value. The advantage to this method is that all of the details relating to starting the process are together in a single place. It makes it much easier to maintain in the future.

Note that I also changed the way the process is started. You used runShellCommandAsynchronously(), which is easier, but brings in some extra overhead. What it does is start an instance of the shell interpreter ash, and then passes the command string to it. ash then parses the command and starts the process. You are actually launching two Linux processes - one for the shell interpreter, and one for Python. By using the syntax I have, it requires some extra lines of code, but it launches the Python interpreter directly - only one process, there is no need to launch the shell interpreter in this case.

Actually, it can be simplified a little more. Add a shebang to the beginning of the Python file, and you can call the file directly. So, add this as the first line of the Python file to let the system know what application to run to process the file:

#!/usr/bin/python

and then you can simply call it like this:

    p.begin("/mnt/sda1/ftp_test.py");
    p.runAsynchronously();

Doing it this way, the system has to first open the file and read the first line to determine what exectuable to launch, but this is offset by eliminating the need for the system to search the environment path looking for the executable. It also allows you to simply run it from the command line by typing ./ftp_test.py which is shorter than typing python every time. (You can't just type ftp_test.py without the leading "./" because unlike MS-DOS, the current folder is not searched when looking for commands - so you have to explicitly tell it to look in the current folder.)

4

Use the global path of the script when calling it from a sketch.

5

ShapeShifter, you're absolutely correct. Running from root, I get an error whenever the file opens:

Traceback (most recent call last):
  File "/mnt/sda1/ftp_test.py", line 123, in <module>
    with open(filePath, "a+") as myFile:
IOError: [Errno 2] No such file or directory: 'tadtest/022616.txt'

Fully qualifying the file path as "/mnt/sda1/tadtest/ftp_test.py" prevents the crash.

I had several more issues with cleaning up rc.local to remove the previous measures I had put in to start the script.

It works perfectly! Thank you so much for your help!

6

Glad to hear you got it working! Thanks for the follow-up, do often we never hear back whether the suggestions helped or not.