Filtering twitter search results.

Hey Everyone,

We are currently trying to filter some information from the JSON dump produced by a sketch that searches twitter. Our sketch is based on a temboo choreo (not sure if this is a dirty topic or not).

We have been messing around with two different choreos. The first one prints the most recent tweet from a specific accounts timeline:

/*
  ReadATweet

  Demonstrates retrieving the most recent Tweet from a user's home timeline 
  using the Temboo Arduino Yun SDK.
  
  This example code is in the public domain.
*/

#include <Bridge.h>
#include <Temboo.h>
#include "TembooAccount.h" // contains Temboo account information

/*** SUBSTITUTE YOUR VALUES BELOW: ***/

// Note that for additional security and reusability, you could
// use #define statements to specify these values in a .h file.
const String TWITTER_ACCESS_TOKEN = "your-twitter-access-token";
const String TWITTER_ACCESS_TOKEN_SECRET = "your-twitter-access-token-secret";
const String TWITTER_API_KEY = "your-twitter-api-key";
const String TWITTER_API_SECRET = "your-twitter-api-secret";

int numRuns = 1;   // execution count, so this doesn't run forever
int maxRuns = 10;   // the max number of times the Twitter HomeTimeline Choreo should run

void setup() {
  Serial.begin(9600);
  
  // For debugging, wait until a serial console is connected.
  delay(4000);
  while(!Serial);
  Bridge.begin();
}
void loop()
{
  // while we haven't reached the max number of runs...
  if (numRuns <= maxRuns) {
    Serial.println("Running ReadATweet - Run #" + String(numRuns++));
    
    TembooChoreo HomeTimelineChoreo;

    // invoke the Temboo client.
    // NOTE that the client must be reinvoked, and repopulated with
    // appropriate arguments, each time its run() method is called.
    HomeTimelineChoreo.begin();
    
    // set Temboo account credentials
    HomeTimelineChoreo.setAccountName(TEMBOO_ACCOUNT);
    HomeTimelineChoreo.setAppKeyName(TEMBOO_APP_KEY_NAME);
    HomeTimelineChoreo.setAppKey(TEMBOO_APP_KEY);

    // tell the Temboo client which Choreo to run (Twitter > Timelines > HomeTimeline)
    HomeTimelineChoreo.setChoreo("/Library/Twitter/Timelines/HomeTimeline");
    
    
    // set the required choreo inputs
    // see https://www.temboo.com/library/Library/Twitter/Timelines/HomeTimeline/
    // for complete details about the inputs for this Choreo

    HomeTimelineChoreo.addInput("Count", "1"); // the max number of Tweets to return from each request
    HomeTimelineChoreo.addInput("AccessToken", TWITTER_ACCESS_TOKEN);
    HomeTimelineChoreo.addInput("AccessTokenSecret", TWITTER_ACCESS_TOKEN_SECRET);
    HomeTimelineChoreo.addInput("ConsumerKey", TWITTER_API_KEY);    
    HomeTimelineChoreo.addInput("ConsumerSecret", TWITTER_API_SECRET);

    // next, we'll define two output filters that let us specify the 
    // elements of the response from Twitter that we want to receive.
    // see the examples at http://www.temboo.com/arduino
    // for more on using output filters
   
    // we want the text of the tweet
    HomeTimelineChoreo.addOutputFilter("tweet", "/[1]/text", "Response");
    
    // and the name of the author
    HomeTimelineChoreo.addOutputFilter("author", "/[1]/user/screen_name", "Response");


    // tell the Process to run and wait for the results. The 
    // return code will tell us whether the Temboo client 
    // was able to send our request to the Temboo servers
    unsigned int returnCode = HomeTimelineChoreo.run();
    
   // a response code of 0 means success; print the API response
    if(returnCode == 0) {
      
      String author; // a String to hold the tweet author's name
      String tweet; // a String to hold the text of the tweet


      // choreo outputs are returned as key/value pairs, delimited with 
      // newlines and record/field terminator characters, for example:
      // Name1\n\x1F
      // Value1\n\x1E
      // Name2\n\x1F
      // Value2\n\x1E      
      
      // see the examples at http://www.temboo.com/arduino for more details
      // we can read this format into separate variables, as follows:
      
      while(HomeTimelineChoreo.available()) {
        // read the name of the output item
        String name = HomeTimelineChoreo.readStringUntil('\x1F');
        name.trim();

        // read the value of the output item
        String data = HomeTimelineChoreo.readStringUntil('\x1E');
        data.trim();

        // assign the value to the appropriate String
        if (name == "tweet") {
          tweet = data;
        } else if (name == "author") {
          author = data;
        }
      }
     
      Serial.println("@" + author + " - " + tweet);
    
    } else {
      // there was an error
      // print the raw output from the choreo
      while(HomeTimelineChoreo.available()) {
        char c = HomeTimelineChoreo.read();
        Serial.print(c);
      }
    }

    HomeTimelineChoreo.close();

  }

  Serial.println("Waiting...");
  delay(90000); // wait 90 seconds between HomeTimeline calls
}

This prints a nice simple string into the serial monitor. Looking like this:

@Authors_name - tweet_content.

We then tried to use the code below to search twitter for a specific term and print it out in the same way. Here is the code:

#include <Bridge.h>
#include <Temboo.h>
#include "TembooAccount.h" // contains Temboo account information, as described below

int numRuns = 1;   // Execution count, so this doesn't run forever
int maxRuns = 10;   // Maximum number of times the Choreo should be executed

void setup() {
  Serial.begin(9600);
  
  // For debugging, wait until the serial console is connected.
  delay(4000);
  while(!Serial);
  Bridge.begin();
}

void loop() {
  if (numRuns <= maxRuns) {
    Serial.println("Running Tweets - Run #" + String(numRuns++));
    
    TembooChoreo TweetsChoreo;

    // Invoke the Temboo client
    TweetsChoreo.begin();
    
    // Set Temboo account credentials
    TweetsChoreo.setAccountName(TEMBOO_ACCOUNT);
    TweetsChoreo.setAppKeyName(TEMBOO_APP_KEY_NAME);
    TweetsChoreo.setAppKey(TEMBOO_APP_KEY);
    
    // Set profile to use for execution
    TweetsChoreo.setCredential("TwitterAccount2");
    
    // Identify the Choreo to run
    TweetsChoreo.setChoreo("/Library/Twitter/Search/Tweets");
    
    // Run the Choreo; when results are available, print them to serial
    TweetsChoreo.run();
    
    while(TweetsChoreo.available()) {
      char c = TweetsChoreo.read();
      Serial.print(c);
    }
    TweetsChoreo.close();
  }

  Serial.println("Waiting...");
  delay(30000); // wait 30 seconds between Tweets calls
}

This prints a whole load of meta data about the tweets it finds.

So we tried to use the filtering system from the home line choreo to produce the nice simple output.

The bit of code we tried to include:

    TweetsChoreo.addOutputFilter("tweet", "/[1]/text", "Response");
    
    TweetsChoreo.addOutputFilter("author", "/[1]/user/screen_name", "Response");

    unsigned int returnCode = TweetsChoreo.run();
    
    if(returnCode == 0) {
      
      String author; 
      String tweet; 
      
      while(TweetsChoreo.available()) {
        String name = TweetsChoreo.readStringUntil('\x1F');
        name.trim();

        String data = TweetsChoreo.readStringUntil('\x1E');
        data.trim();

        // assign the value to the appropriate String
        if (name == "tweet") {
          tweet = data;
        } else if (name == "author") {
          author = data;

With this section it compiles and runs fine with no errors. However, it prints a blank statment like this:

@ -

With no author name or tweet content.

We have spent a good while trying different things with no result and are starting to pull our hair out. I hope this all makes sense and thank you to anyone who has any suggestions!

EDIT: We are using the Arduino Yun!

Looking at the difference between the example responses in the docs for statuses/user_timeline and search/tweets , it looks like your filters in the second case need to be /statuses/[1]/text and /statuses/[1]/user/screen_name, instead of just /[1]/text and /[1]/user/screen_name like they are for timeline.

Replying here as well as on reddit just so others know where we're at.

hobbified's suggestion didn't change anything. However, we are reading over the links he sent. So finger crossed they might help clear the air :slight_smile:

I believe since you need the first item in the 'statuses' array, this is the path you want:

TweetsChoreo.addOutputFilter("tweet", "/statuses[1]/text", "Response");

TweetsChoreo.addOutputFilter("author", "/statuses[1]/user/screen_name", "Response");

That makes sense. Little XPath fail on my part.