Incomplete Temboo Response using Yahoo Weather Choreo Output Filters

Hardware Arduino Yún
1

Hi All,

I have spent many hours trying to solve this problem. Using the GetWeather choreo returns a complete response if no output filters are used. Trouble is, output filters are nice for grabbing the data I am interested in, e.g.: current temp and humidity, forecasted highs and lows, and the weather code for forecasted conditions (to later convert to text, e.g. "Thundershowers").

So, the issue is that when I use multiple output filters (~5), the response is incomplete. Some of the attributes I have requested from the XML file are not returned. I thought this might be an Arduino RAM issue, so I have checked the free memory, but it is unclear to me whether or not it is running out of RAM while sending the Temboo process to Linino. I shortened the XML paths and labels in an effort to use less memory, which has appeared to let me display more output filters (e.g. only ~1 with full XML paths, but ~2-3 with simplified paths). If I switch the order of the .OutputFilters in the source it appears to pick up the first couple and omit the rest. Is there a way to check the Process command that the TembooChoreo class instance is storing at a given time?

Any ideas on how to diagnose this problem further would be much appreciated. I am sure it is not a unique problem... Thanks in advance.

Console output:

Free Memory @ Start = 867
Free Memory pre-run = 330
Free Memory post-run = 594
t
24
lT
14
hT
26
HTTP_CODE
200

Temboo Source Sketch:

void getWeather(){
  TembooChoreo WeatherChoreo;
  WeatherChoreo.begin();
  WeatherChoreo.setAccountName(TEMBOO_ACCOUNT);
  WeatherChoreo.setAppKeyName(TEMBOO_APP_KEY_NAME);
  WeatherChoreo.setAppKey(TEMBOO_APP_KEY);
  WeatherChoreo.setChoreo("/Library/Yahoo/Weather/GetWeather");
//  WeatherChoreo.addInput("Address", ADDRESS_FOR_FORECAST);
  WeatherChoreo.addInput("WOEID", "24203543");
  WeatherChoreo.addInput("Units", "c");
  WeatherChoreo.addOutputFilter("lT", "//yweather:forecast[1]/@low", "Response");
  WeatherChoreo.addOutputFilter("hT", "//yweather:forecast[1]/@high", "Response");
//  WeatherChoreo.addOutputFilter("f", "//yweather:forecast[1]/@code", "Response");
  WeatherChoreo.addOutputFilter("h", "//yweather:atmosphere/@humidity", "Response");  
  WeatherChoreo.addOutputFilter("t", "//yweather:condition/@temp", "Response");
//  WeatherChoreo.addOutputFilter("c", "//yweather:condition/@code", "Response");

  Console.print("Free Memory pre-run = ");
  Console.println(freeMemory());
    
  Console.print("Run code = ");
  Console.println(WeatherChoreo.run());

  Console.print("Free Memory post-run = ");
  Console.println(freeMemory());

  while(WeatherChoreo.available()){
    String filter = WeatherChoreo.readStringUntil('\x1F');
    Console.print(filter);
    filter.trim(); //removes newlines

    String data = WeatherChoreo.readStringUntil('\x1E');
    Console.print(data);
    data.trim(); //removes newlines
  
    if (filter == "t"){ambTemp = data.toInt();}
    else if (filter == "h"){ambHumi = data.toInt();}
    else if (filter == "hT"){hiTemp = data.toInt();}
    else if (filter == "lT"){lowTemp = data.toInt();}
    else if (filter == "c"){code = data.toInt();}
    else if (filter == "f"){forecast = data.toInt();}
  }
  WeatherChoreo.close();
}
2

Almost certainly a memory issue... Adding a few more freeMemory() calls to the code, I can see that adding the first two OutputFilters uses about 42 bytes each, and after those two the memory only goes drops by 1 byte for each of the next AddOutputFilter attempts. So is there a bottom limit on RAM for certain methods (e.g. ~330 bytes) or is the TembooChoreo class method of AddOutputFilter not working properly (e.g. bug)?

Console:

Free Memory @ Start = 833
422
380
337
336
335
Free Memory pre-run = 335
Run code = 0
Free Memory post-run = 560
lT
14
hT
26
HTTP_CODE
200

void getWeather(){
  TembooChoreo WeatherChoreo;
  WeatherChoreo.begin();
  WeatherChoreo.setAccountName(TEMBOO_ACCOUNT);
  WeatherChoreo.setAppKeyName(TEMBOO_APP_KEY_NAME);
  WeatherChoreo.setAppKey(TEMBOO_APP_KEY);
  WeatherChoreo.setChoreo("/Library/Yahoo/Weather/GetWeather");
//  WeatherChoreo.addInput("Address", ADDRESS_FOR_FORECAST);
  WeatherChoreo.addInput("WOEID", "24203543");
  WeatherChoreo.addInput("Units", "c");
//  Console.print("Free Memory add 0 = ");
  Console.println(freeMemory());
  WeatherChoreo.addOutputFilter("lT", "//yweather:forecast[1]/@low", "Response");
//  Console.print("Free Memory add 1 = ");
  Console.println(freeMemory());
  WeatherChoreo.addOutputFilter("hT", "//yweather:forecast[1]/@high", "Response");
//  Console.print("Free Memory add 2 = ");
  Console.println(freeMemory());
  WeatherChoreo.addOutputFilter("t", "//yweather:condition/@temp", "Response");
//  Console.print("Free Memory add 3 = ");
  Console.println(freeMemory());
  WeatherChoreo.addOutputFilter("h", "//yweather:atmosphere/@humidity", "Response"); 
//    Console.print("Free Memory add 4 = ");
  Console.println(freeMemory());
//  WeatherChoreo.addOutputFilter("f", "//yweather:forecast[1]/@code", "Response");
 
//  WeatherChoreo.addOutputFilter("c", "//yweather:condition/@code", "Response");

  Console.print("Free Memory pre-run = ");
  Console.println(freeMemory());
    
  Console.print("Run code = ");
  Console.println(WeatherChoreo.run());

  Console.print("Free Memory post-run = ");
  Console.println(freeMemory());

  while(WeatherChoreo.available()){
    char c = WeatherChoreo.read();
    Console.print(c);
  }

  while(WeatherChoreo.available()){
    String filter = WeatherChoreo.readStringUntil('\x1F');
    Console.print(filter);
    filter.trim(); //removes newlines

    String data = WeatherChoreo.readStringUntil('\x1E');
    Console.print(data);
    data.trim(); //removes newlines
    Console.print("Free Memory filter = ");
    Console.println(freeMemory());
  
    if (filter == "t"){ambTemp = data.toInt();}
    else if (filter == "h"){ambHumi = data.toInt();}
    else if (filter == "hT"){hiTemp = data.toInt();}
    else if (filter == "lT"){lowTemp = data.toInt();}
    else if (filter == "c"){code = data.toInt();}
    else if (filter == "f"){forecast = data.toInt();}
  }
  WeatherChoreo.close();
}
3 
opkg update
opkg install python-expat
opkg install python-openssl #adds ssl support to python
opkg install distribute #it contains the easy_install command line tool (this can take some time)
easy_install pip #installs pip  (this can take some time)
pip install pywapi --allow-external pywapi --allow-unverified pywapi

https://code.google.com/p/python-weather-api/

nano /mnt/sda1/pyw.py

#!/usr/bin/python
import  pywapi
import string
import sys
sys.path.insert(0, '/usr/lib/python2.7/bridge/')
from bridgeclient import BridgeClient as bridgeclient
yahoo_result = pywapi.get_weather_from_yahoo('10001')
print "Yahoo says: It is " + string.lower(yahoo_result['condition']['text']) + " and " + yahoo_result['condition']['temp'] + "C now in New York.\n"
value = bridgeclient()
value.put('t',yahoo_result['condition']['temp'])
print value.get('t')

chmod 755 /mnt/sda1/pyw.py

Test python code at teminal:

root@Arduino:/mnt/sda1# /mnt/sda1/pyw.py
Yahoo says: It is fair and 28C now in New York.

28

Arduino code:

/*
  Running process using Process class.

 This sketch demonstrate how to run linux processes
 using an Arduino Yún.

 created 5 Jun 2013
 by Cristian Maglie

 This example code is in the public domain.

 http://arduino.cc/en/Tutorial/Process

 */

#include <Process.h>

void setup() {
  Bridge.begin();
   // Initialize Serial
  Serial.begin(9600);

  // Wait until a Serial Monitor is connected.
  while (!Serial);
}

void loop() {
  // Do nothing here.
  Process p;              
  p.begin("/mnt/sda1/pyw.py");      
//p.addParameter("--connect-timeout");
  p.run();
   // Print command output on the Serial.
  // A process output can be read with the stream methods
  while (p.available() > 0) {
    char c = p.read();
    Serial.print(c);
  }
  // Ensure the last bit of data is sent.
  Serial.flush();
  char temp[10];
  Bridge.get("t", temp, sizeof(temp));
  Serial.print(temp);
  delay(5000);
}
4

Hi Sonnyyu,

While I agree that not using the Temboo wrapper would be preferable (less efficient - perhaps the problem, and limited # of calls per month for free), I was implementing my solution with Temboo in the hopes of it being quick and easy! Turns out it wasn't... but I'm still curious to understand the issues I am having. Not saying I won't be implementing your great suggestion though...!

Thanks for the input. Any explanation on the issue I am having?

5

While I can't help with your issue, I would like to point out that temboo has a python sdk as well. So you can call temboo services and do all the filtering you wish on the linux side, and have the arduino side happily consume only the interesting data.

This is an example of using the SDK for uploading a photo on facebook http://dev.mikamai.com/post/73613525813/arduino-yun-social-photo-camera

6

[quote author=Federico Fissore link=topic=264822.msg1868172#msg1868172 date=1409814504]
While I can't help with your issue, I would like to point out that temboo has a python sdk as well. So you can call temboo services and do all the filtering you wish on the linux side, and have the arduino side happily consume only the interesting data.[/quote]

Thanks for the input! I agree that moving this to the Linux side is the best solution, and also for any future additions. I wasn't aware that Temboo had a Python SDK... too fixated on "Arduino" ;).