Parse JSON, Arduino Yun

Hi Guys,

I am currently attempting to program a door switch (Reed switch) that will set a Nest Thermostat to "away" when a door is opened and return it to "Home" when the door is closed again.

You can use a REST API which I am currently attempting to interact with, (i can do this manually via Curl)

I have an Arduino Yun on order and have started to write the sketch but am having some problems. I have the basic things down like the set away/home action as that will not change, what I would like to do is automate the program so it can obtain the structure_ID and eventually the auth token.

As obtaining the structure_ID seems to be the smaller problem I am working on that first and will be working backwards to obtain the access token which for the moment is hard coded (Nest use static tokens)

Currently the getStructureID sends a GET request via the bridge using Curl (Command as shown below)

curl -L -X GET -H "Accept: application/json" "https://developer-api.nest.com/?auth=AUTH TOKEN"

This then returns the following JSON Response, from that I would like to filter out the structure_id value and enter it into the String structureID.

Now I have been looking round and have found the aJSON library and have no idea how to use it and it seems like it wants the whole JSON response cached in an array which will send up somewhere around 1200bytes which is most of the memory I have available on the Yun.

Is there a way to parse this on the fly say write to a small buffer say 200 chars and if that does not contain the data that we require clear the buffer and load the next set of data and filter until we find the value that we want?

  "devices": {
    "thermostats": {
      "C2U421cfM3DmjD3VtJxTMYbYYj9JDwkY": {
        "humidity": 50,
        "locale": "en-GB",
        "temperature_scale": "C",
        "is_using_emergency_heat": false,
        "has_fan": false,
        "software_version": "4.3.1",
        "has_leaf": true,
        "device_id": "C2U421cfM3DmjD3VtJxTMYbYYj9JDwkY",
        "name": "Basement",
        "can_heat": true,
        "can_cool": false,
        "hvac_mode": "heat",
        "target_temperature_c": 21.0,
        "target_temperature_f": 70,
        "target_temperature_high_c": 24.0,
        "target_temperature_high_f": 75,
        "target_temperature_low_c": 20.0,
        "target_temperature_low_f": 68,
        "ambient_temperature_c": 21.0,
        "ambient_temperature_f": 70,
        "away_temperature_high_c": 24.0,
        "away_temperature_high_f": 76,
        "away_temperature_low_c": 12.5,
        "away_temperature_low_f": 55,
        "structure_id": "rz5wmnglIbLGRSIkONOIPddlthddddIzHESHIVddddlx2xcBqk7y9A",
        "fan_timer_active": false,
        "name_long": "Basement Thermostat",
        "is_online": true
        }
      }
    },

  "structures": {
    "rz5wmnglIbLGRSIkONOIPddlthddddIzHESHIVddddlx2xcBqk7y9A": {
      "name": "Home",
      "country_code": "GB",
      "time_zone": "Europe/London",
      "away": "home",
      "thermostats": ["C2U421cfM3DmjD3VtJxTMYbYYj9JDwkY"],
      "structure_id": "rz5wmnglIbLGRSIkONOIPddlthddddIzHESHIVddddlx2xcBqk7y9A"
      }
    },

  "metadata": {
    "access_token": "Token ID",
    "client_version": 1
    }
  }
#include <Process.h>

const int buttonPin = 2;
const int ledPin =  8;
const String accessToken = "Some Token ID";
// const String structureID = "Some Structure ID";

int buttonState = 0;
String awayState = "home";
String structureID;


void setup() {
  Bridge.begin();
  Serial.begin(9600);
  while (!Serial);

  pinMode(ledPin, OUTPUT);
  pinMode(buttonPin, INPUT);

  buttonState = digitalRead(buttonPin);
  if (buttonState = HIGH) {
    String awayState = "home";
  } else; {
    String awayState = "away";
  }

  Serial.println('Away State = ' + awayState);

}

void getStructureID() {
  Process getStructureID;
  getStructureID.begin("curl");
  getStructureID.addParameter("-L");
  getStructureID.addParameter("-X");
  getStructureID.addParameter("GET");
  getStructureID.addParameter("-H");
  getStructureID.addParameter("\"Accept: application/json\"");
  getStructureID.addParameter("\"https://developer-api.nest.com/?auth=" + accessToken + "\"");
  getStructureID.run();

  while (getStructureID.available() > 0) {
    char c = getStructureID.read();
    Serial.print(c);

  }

  Serial.flush();

}

void setAway() {
  Process setAway;
  setAway.begin("curl");
  setAway.addParameter(" - v");
  setAway.addParameter(" - L");
  setAway.addParameter(" - X");
  setAway.addParameter("PUT");
  setAway.addParameter("\"https://developer-api.nest.com/structures/" + structureID + "?auth=" + accessToken + "\"");
  setAway.addParameter("-H");
  setAway.addParameter("\"Content-Type: application/json\"");
  setAway.addParameter("-d");
  setAway.addParameter("{\"away:\"" + awayState + "\"}");
  setAway.run();

  while (setAway.available() > 0) {
    char c = setAway.read();
    Serial.print(c);
  }

  Serial.flush();

}

void loop() {
  buttonState = digitalRead(buttonPin);
  if (buttonState == HIGH) {
    digitalWrite(ledPin, LOW);
    String awayState = "home";
  }
  else {
    digitalWrite(ledPin, HIGH);
    String awayState = "away";
  }

  Serial.println('Away State = ' + awayState);
  setAway();

}

NOTE: I have not run this code on actual hardware yet as my Arduino is still in the post so this sketch may net even do what I want it to do or expect it to do but am pretty sure that the setAway method should work ok.

Any pointers on how I can improve my code are also appreciated.

I have found this Forum post that suggests parsing this on the Linux side and passing the data string over bridge to the Arduino now I have no idea how I could pass the response from Curl to a Python script without going over the bridge twice. Again any pointers would be great.

http://forum.arduino.cc/index.php?topic=192626.0

There's a subforum for the Yun, perhaps moderator could move this thread there.

Ive done some more playing and can get closer to what I want with terminal commands.

eg

curl -s -L -X GET -H "Accept: application/json" "https://developer-api.nest.com/?auth=ACCESS TOKEN" | python -mjson.tool | grep structure_id

Outputs the following which can be trimmed to store the value as a String.

            "structure_id": "rz5wmnglIbLGRSIkONOIPddlthddddIzHESHIVddddlx2xcBqk7y9A",
            "structure_id": "rz5wmnglIbLGRSIkONOIPddlthddddIzHESHIVddddlx2xcBqk7y9A",

However this looks like a messy solution.

Any tips?

Any tips?

There was one in reply #1 that you ignored.

PaulS:
There was one in reply #1 that you ignored.

How so? Mark let me know there was a Yun forum that this question may be better suited in, this has been noted for any future posts. However I cannot move this thread myself so would have to wait for a mod to do this.

so would have to wait for a mod to do this.

Have you asked one of them to?

Thread moved to more appropriate section.

warrmr:
Hi Guys,

I am currently attempting to program a door switch (Reed switch) that will set a Nest Thermostat to "away" when a door is opened and return it to "Home" when the door is closed again.

You can use a REST API which I am currently attempting to interact with, (i can do this manually via Curl)

I have an Arduino Yun on order and have started to write the sketch but am having some problems. I have the basic things down like the set away/home action as that will not change, what I would like to do is automate the program so it can obtain the structure_ID and eventually the auth token.

::::SNIP::::

I have found this Forum post that suggests parsing this on the Linux side and passing the data string over bridge to the Arduino now I have no idea how I could pass the response from Curl to a Python script without going over the bridge twice. Again any pointers would be great.

Parsing JSON on Arduino Yún - Arduino Yún - Arduino Forum

warrmr,
nest.com is using the service from firebase.com It's quit possible you could get your information by calling for it directly.

so instead of this

https://developer-api.nest.com/?auth=AUTH TOKEN

this might work:

https://developer-api.nest.com/devices/thermostats/C2U421cfM3DmjD3VtJxTMYbYYj9JDwkY/structure_id/.json?auth=AUTH TOKEN

If not I could help you get it working with python. However, there is a good change it can be done with HTTPClient or YunClient

It's late. I'll be back in 12-24 hours.

Jesse

Concept only, has no hardware.

Install software:

opkg update
opkg install php5-cli
opkg install php5-mod-curl

Php code:

nano /mnt/sda1/rest_thermostat.php
#!/usr/bin/php-cli
<?php
require_once('nest.class.php');
// Your Nest username and password.
define('USERNAME', 'you@gmail.com');
define('PASSWORD', 'Something other than 1234 right?');
$nest = new Nest();
// Set Away mode
$nest->setAway(TRUE);
// Turn off Away mode
//$nest->setAway(FALSE);
cd  /mnt/sda1
wget https://raw.githubusercontent.com/gboudreau/nest-api/master/nest.class.php --no-check-certificate
chmod 755 /mnt/sda1/rest_thermostat.php

Test setAway function alone:

/mnt/sda1/rest_thermostat.php

Arduino code:

#include <Process.h>
// digital pin X has a pushbutton attached to it. Give it a name:
int doorswitch = X;
void setup() {
 Bridge.begin();  // Initialize Bridge
 // make the pushbutton's pin an input:
  pinMode(doorswitch, INPUT);
}
void loop() {
  // read the input pin:
  int doorswitchState = digitalRead(doorswitch);
if (doorswitchState) {
 Process p;              
 p.begin("/mnt/sda1/rest_thermostat.php");      
 p.run();
}
 delay(5000); 
}

I think sonnyyu solution is the best, because you let the linux side doing all the networking while the Arduino code is doing only the control part.

If you want you can use python instead of php if you are more faimiliar with. Then you can launch the python script instead of the php script from the sketch.

Thanks Guys,

@Jessie, Thankyou.

I can do what I want by sending this command via Curl from my mac.

curl -v -L -X PUT "https://developer-api.nest.com/structures/STRUCTURE_ID?auth=AUTH TOKEN" -d '{"away":"away"}'

Obviously swapping AUTH TOKEN and STRUCTURE_ID for the appropriate values.

The structure_ID is obtained by running the GET command swapping AUTH TOKEN for the actual token.

curl -L -X GET -H "Accept: application/json" "https://developer-api.nest.com/?auth=.AUTH TOKEN"

As a bare bones program that does the job I am happy that it will work, however if the auth token changes or the structure ID changes the program will stop working so I would like to obtain these values programmatically starting with the easiest one to obtain STRUCTURE_ID.

The AUTH Token is obtained by going to this URL

https://api.home.nest.com/oauth2/access_token?client_id=d712052d-13f3-45b8-8c47-ee0970dadc94&code=AUTHORIZATION_CODE&client_secret=0mHEsnQusqB7mPk0kTNlX7YGY&grant_type=authorization_code

Agreeing to give my program access to temp and away status then logging in this will then give you a PIN which is exchanges for the token by sending this request.

curl -X POST "https://api.home.nest.com/oauth2/access_token?client_id=d712052d-13f3-45b8-8c47-ee0970dadc94&code= PIN&client_secret=0mHEsnQusqB7mPk0kTNlX7YGY&grant_type=authorization_code"

swapping PIN for the actual PIN provided by the Nest website.

Doing things this way means that my program will never need my actual Nest login so can never send it in clear text across the internet (paranoia).

Order of operations outlined here.

@Sonnyy thank you.
I don't really understand PHP but that on github looks to do what I want, but I don't like the way that it authenticates with Nest (password in clear text on within the code) so I think I will use that as a guide and continue working on my own.

I did start hacking together something in Python last night and have successfully managed to get the Strucutre_ID from the JSON. So working backwards I would need to split this out into modules and work on the authentication piece and port over the away/home state module from the sketch.

import urllib2
import urllib
import json

data = {}
data['auth'] = 'Token Here'
url_values = urllib.urlencode(data)
url =  'https://developer-api.nest.com/'
fullURL = url + '?' + url_values

req = urllib2.Request(fullURL)
req.add_header('Accept', 'application/json')
response = urllib2.urlopen(req)
jsonResponse = response.read()
 
try:
    decoded = json.loads(jsonResponse)
    dataStructure = decoded['structures']
    
    for item in dataStructure:
      print decoded['structures'][item]['structure_id']
 
except (ValueError, KeyError, TypeError):
    print "JSON format error"

Plan B (dirty and quick):

opkg update
opkg install curl
nano /mnt/sda1/setaway.sh
#!/bin/ash
/usr/bin/curl -v -L -X PUT "https://developer-api.nest.com/structures/STRUCTURE_ID?auth=AUTH TOKEN" -d '{"away":"away"}'
chmod 755 /mnt/sda1/setaway.sh

Test:

/mnt/sda1/setaway.sh

Plan C ( python, libcurl way, Hardcode STRUCTURE_ID):

Since curl is user interface of libcurl, we could installed PycURL which is a Python interface to libcurl.

opkg update
opkg install python-curl

Translate:

curl -L -X GET -H "Accept: application/json" "https://developer-api.nest.com/?auth=.AUTH TOKEN"

curl -v -L -X PUT "https://developer-api.nest.com/structures/STRUCTURE_ID?auth=AUTH TOKEN" -d '{"away":"away"}'

into python/pycurl code.

PycURL is very fast compare with urllib2, and relatively thin layer over libcurl. This means it has a somewhat steep learning curve unless you are already familiar with libcurl's C API.

If you got this error:

curl https://www.apple.com
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

The fix is here:

wget -O ca-certificates_20140325_ar71xx.ipk https://www.dropbox.com/s/rmvkunea1ifa9zn/ca-certificates_20140325_ar71xx.ipk?dl=0 --no-check-certificate
opkg update
opkg install openssl-util
opkg install ca-certificates_20140325_ar71xx.ipk

Thanks for all of the help, I have most of this setup so figure I would post the code of any one in the future and also see if I can get some pointers with my Python Script.

Python Code located at /root/setAway.py

#!/usr/bin/python

import os
import sys
import json
import pycurl
import urllib
import os.path
import cStringIO

PATH = '/root/authToken'
accessToken = 0
structureId = 0
verbose = False
awayState = 0

buffer = cStringIO.StringIO()

def parseArguments():
	global verbose
	global awayState
	counter = 0

	for eachArg in sys.argv:
		if eachArg == "-h" or eachArg == "--home":
			# print "Away State: Home"
			awayState = "home"
			counter = counter + 1
		elif eachArg == "-a" or eachArg == "--away":
			# print "Away State: Away"
			awayState = "away"
			counter = counter + 1
		elif eachArg == "-v" or eachArg == "--verbose":
			verbose = True

	if awayState == 0 or counter > 1:
		sys.exit("Away State not valid: Please specify --home or --away")

def getAccessToken():
	global accessToken
	print "Please authenticate with Nest to obtain PIN, "
	print
	print "https://home.nest.com/login/oauth2?client_id=d712052d-13f3-45b8-8c47-ee0970dadc94&state=STATE"
	print
	pin = raw_input("Please enter Nest PIN? ")

	url = {'client_id' : 'd712052d-13f3-45b8-8c47-ee0970dadc94', 'code' : pin, 'client_secret' : '0mHEsnQusqB7mPk0kTNlX7YGY', 'grant_type' : 'authorization_code'}
	encodedUrl = urllib.urlencode(url)

	getToken = pycurl.Curl()
	getToken.setopt(getToken.URL, 'https://api.home.nest.com/oauth2/access_token')
	getToken.setopt(getToken.CONNECTTIMEOUT, 5)
	getToken.setopt(getToken.TIMEOUT, 8)
	getToken.setopt(getToken.FAILONERROR, True)
	getToken.setopt(getToken.POSTFIELDS, encodedUrl)
	getToken.setopt(getToken.VERBOSE, verbose)
	getToken.setopt(getToken.WRITEFUNCTION, buffer.write)
	getToken.perform()

	jsonResponse = buffer.getvalue()

	try:
		decoded = json.loads(jsonResponse)
		accessToken = decoded['access_token']
		print "Access token: " + accessToken
		tokenFile = open(PATH, "w")
		tokenFile.write(accessToken + "\O")
		tokenFile.close()

	except (ValueError, KeyError, TypeError):
		decoded = json.loads(jsonResponse)
		errorDescription = "Error: " + decoded['error_description']
		print errorDescription

def loadAccessToken():
	if os.path.isfile(PATH) and os.access(PATH, os.R_OK):
		global accessToken
		# print "Loading Access Token..."
		tokenFile = open(PATH, "r")
		accessToken = tokenFile.read()
		tokenFile.close()
		accessToken = accessToken.rstrip('\n')
		# print "Access Token is: " + accessToken
	else:
		print "Access Token not found!"
		print
		getAccessToken();

def getStructureID():
	global structureId
	url = {'auth' : accessToken}
	encodedUrl = urllib.urlencode(url)

	getStructureID = pycurl.Curl()
	getStructureID.setopt(getStructureID.URL, 'https://developer-api.nest.com/?' + encodedUrl)
	getStructureID.setopt(getStructureID.CONNECTTIMEOUT, 5)
	getStructureID.setopt(getStructureID.TIMEOUT, 8)
	getStructureID.setopt(getStructureID.FAILONERROR, True)
	getStructureID.setopt(getStructureID.HTTPHEADER, ['Accept: application/json'])
	getStructureID.setopt(getStructureID.VERBOSE, verbose)
	getStructureID.setopt(getStructureID.WRITEFUNCTION, buffer.write)
	getStructureID.setopt(getStructureID.FOLLOWLOCATION, 2)
	getStructureID.perform()

	jsonResponse = buffer.getvalue()

	try:
		decoded = json.loads(jsonResponse)
		dataStructure = decoded['structures']
		for item in dataStructure:
			structureId = decoded['structures'][item]['structure_id']

	except (ValueError, KeyError, TypeError):
		decoded = json.loads(jsonResponse)
		errorDescription = "Error: " + decoded['error_description']
		print errorDescription

def setAway():
	url = {'auth' : accessToken}
	encodedUrl = urllib.urlencode(url)

	setAway = pycurl.Curl()
	setAway.setopt(setAway.URL, 'https://developer-api.nest.com/structures/' + structureId.encode('iso-8859-1') + "?" + encodedUrl)
	setAway.setopt(setAway.CONNECTTIMEOUT, 5)
	setAway.setopt(setAway.TIMEOUT, 8)
	setAway.setopt(setAway.FAILONERROR, True)
	setAway.setopt(setAway.CUSTOMREQUEST, "PUT")
	setAway.setopt(setAway.POST, 1)
	setAway.setopt(setAway.POSTFIELDS, '{"away":"' + awayState + '"}')
	setAway.setopt(setAway.HTTPHEADER, ['Accept: application/json'])
	setAway.setopt(setAway.VERBOSE, verbose)
	setAway.setopt(setAway.WRITEFUNCTION, buffer.write)
	setAway.setopt(setAway.FOLLOWLOCATION, 2)
	setAway.perform()

	jsonResponse = buffer.getvalue()

	print "Away State: " + awayState

parseArguments()
loadAccessToken()
getStructureID()
setAway()

Arduino Code

#include <Process.h>

// set pin numbers:
const int buttonPin = 2;    
const int ledPin = 13;      

// Declare Global Variables
int ledState = LOW;         
int buttonState;            
int lastButtonState = HIGH;   
String awayState = "-a -v";
long lastDebounceTime = 0;  
long debounceDelay = 10000;    // Wait 10s to see if door is closed again

void setup() {
  Bridge.begin(); 
  Serial.begin(9600);
  while (!Serial);

  pinMode(buttonPin, INPUT_PULLUP); // Enable internal pullup
  pinMode(ledPin, OUTPUT);

  buttonState = digitalRead(buttonPin);  // Read button state and set LED as appropriate (On = Door Open)
  if (buttonState == HIGH) {
    ledState = HIGH;
  }

  digitalWrite(ledPin, ledState);
  Serial.println("Ready: .. ");

}

void setAwayState() {
  Serial.println("Away State: " + awayState);

  Process p;
  p.runShellCommand("python /root/setAway.py " + awayState);
  p.run();

  while (p.available() > 0) {
    char c = p.read();
    Serial.print(c);
  }
  
  Serial.flush();
  digitalWrite(ledPin, ledState);

}


void loop() {
  int reading = digitalRead(buttonPin);

  if (reading != lastButtonState) {
    // reset the debouncing timer
    lastDebounceTime = millis();
  }

  if ((millis() - lastDebounceTime) > debounceDelay) { // If the door has been in the current state for 10s setAwayState

    if (reading != buttonState) {
      buttonState = reading;
      ledState = !ledState;
      if (buttonState == HIGH) {
        Serial.println("Door is open!");
        awayState = "-a -v";
        setAwayState();
      } else {
        Serial.println("Door is closed!");
        awayState = "-h -v";
        setAwayState();
      }

    }
  }

  lastButtonState = reading;
}

If the door is open and closed within 10 seconds the program will do nothing otherwise it will set the thermostat to away until the door is closed again. The issue that I get is that Pycurl doesn't seem to pass the verbose debug to the Serial Console the only output is seen below.

This isn't a major show stopper but I would like to know how I can output the verbose output via the serial console as it takes around 3 seconds for the Python script to run and finish setting away.

Serial Monitor

Ready: .. 
Door is open!
Away State: -a -v
Away State: away
Door is closed!
Away State: -h -v
Away State: home
Door is open!
Away State: -a -v
Away State: away
Door is closed!
Away State: -h -v
Away State: home

This is what I would expect to see on the serial monitor when sending the Verbose switch, rather than the print statements in the Python file.

* Hostname was NOT found in DNS cache
*   Trying 54.243.162.162...
* Connected to developer-api.nest.com (54.243.162.162) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: developer-api.nest.com
* Server certificate: Go Daddy Secure Certificate Authority - G2
* Server certificate: Go Daddy Root Certificate Authority - G2
> GET /?auth=TOKEN HTTP/1.1
User-Agent: PycURL/7.19.5 libcurl/7.37.1 SecureTransport zlib/1.2.5
Host: developer-api.nest.com
Accept: application/json

< HTTP/1.1 307 Temporary Redirect
< Content-Type: application/json; charset=UTF-8
< Access-Control-Allow-Origin: *
< Cache-Control: private, no-cache, max-age=0
< Location: https://firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553/?auth=TOKEN
< Connection: close
< Content-Length: 0
<
* Closing connection 0
* Issue another request to this URL: 'https://firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553/?auth=TOKEN'
* Hostname was NOT found in DNS cache
*   Trying 174.129.167.4...
* Connected to firebase-apiserver06-tah01-iad01.dapi.production.nest.com (174.129.167.4) port 9553 (#1)
* TLS 1.2 connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.dapi.production.nest.com
* Server certificate: Go Daddy Secure Certificate Authority - G2
* Server certificate: Go Daddy Root Certificate Authority - G2
> GET /?auth=TOKEN HTTP/1.1
User-Agent: PycURL/7.19.5 libcurl/7.37.1 SecureTransport zlib/1.2.5
Host: firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553
Accept: application/json

< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Access-Control-Allow-Origin: *
< Cache-Control: private, no-cache, max-age=0
< Connection: close
< Content-Length: 1300
<
* Closing connection 1
* Hostname was NOT found in DNS cache
*   Trying 184.73.153.254...
* Connected to developer-api.nest.com (184.73.153.254) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
* Server certificate: developer-api.nest.com
* Server certificate: Go Daddy Secure Certificate Authority - G2
* Server certificate: Go Daddy Root Certificate Authority - G2
> PUT /structures/rz5wmnglIbLGRSIkONOIPddlthSFcVIzHESHIV2Psilx2xcBqk7y9A?auth=TOKEN HTTP/1.1
User-Agent: PycURL/7.19.5 libcurl/7.37.1 SecureTransport zlib/1.2.5
Host: developer-api.nest.com
Accept: application/json
Content-Length: 15
Content-Type: application/x-www-form-urlencoded

* upload completely sent off: 15 out of 15 bytes
< HTTP/1.1 307 Temporary Redirect
< Content-Type: application/json; charset=UTF-8
< Access-Control-Allow-Origin: *
< Cache-Control: private, no-cache, max-age=0
< Location: https://firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553/structures/rz5wmnglIbLGRSIkONOIPddlthSFcVIzHESHIV2Psilx2xcBqk7y9A?auth=TOKEN
< Connection: close
< Content-Length: 0
<
* Closing connection 0
* Issue another request to this URL: 'https://firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553/structures/rz5wmnglIbLGRSIkONOIPddlthSFcVIzHESHIV2Psilx2xcBqk7y9A?auth=TOKEN'
* Hostname was NOT found in DNS cache
*   Trying 174.129.167.4...
* Connected to firebase-apiserver06-tah01-iad01.dapi.production.nest.com (174.129.167.4) port 9553 (#1)
* TLS 1.2 connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.dapi.production.nest.com
* Server certificate: Go Daddy Secure Certificate Authority - G2
* Server certificate: Go Daddy Root Certificate Authority - G2
> PUT /structures/rz5wmnglIbLGRSIkONOIPddlthSFcVIzHESHIV2Psilx2xcBqk7y9A?auth=TOKEN HTTP/1.1
User-Agent: PycURL/7.19.5 libcurl/7.37.1 SecureTransport zlib/1.2.5
Host: firebase-apiserver06-tah01-iad01.dapi.production.nest.com:9553
Accept: application/json
Content-Length: 15
Content-Type: application/x-www-form-urlencoded

* upload completely sent off: 15 out of 15 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Access-Control-Allow-Origin: *
< Cache-Control: private, no-cache, max-age=0
< Connection: close
< Content-Length: 15
<
* Closing connection 1
Away State: home

Any notes on how I could do this better would be much appreciated.

warrmr:
Thanks for all of the help, I have most of this setup so figure I would post the code of any one in the future and also see if I can get some pointers with my Python Script.

::::SNIP::::

This isn't a major show stopper but I would like to know how I can output the verbose output via the serial console as it takes around 3 seconds for the Python script to run and finish setting away.

::::SNIP::::

warrmr,
it's quite likely the python is writing to "standard error" and not "standard output". You may be able to redirect that with the library, or with python redirect the pipe to your console.

Google: Pycurl redirect stderr
Google: python redirect stderr

This looks like it: Callbacks - PycURL 7.19.5.1 documentation

Jesse