Hello Folks,
I think this is the first time I have ever posted to a forum. So if I am doing something wrong, please let me know. I haven't written code in over 15 years and now I have a IoT project and no resources to complete. I have a bunch of legacy equipment that is RS232 based, I need to pull information from the unit and then do two things. 1. Use that data and display it on a locally hosted web page as close to real time as possible and 2) Send that data to a database server out on the cloud every hour or so.
Heres how I proceeded with the setup:
- Arduino Yun
-- I added an SD card to the system per the instructions - FTDI USB to RS232 cable to legacy unit
- I really have no need for the Arduino side except for maybe some status indicators, maybe console for debugging, who knows
I wrote a python program that polls the legacy equipment and was using puts into the key/value datastore.
I then I wrote a index.html file which I placed in the /mnt/sda1/project/www folder
I used bootstrap.js for the formatting of the page and to provide a responsive web page which will allow the data formatting to change based on the device.
So the script on the arduino side, basically just starts a project.py script and lets it run asynchronously. I was eventually going to place this in the startup process on the linino side. Currently I just start it from a ssh CLI. The project.py opens the USB port, queries the device, and then puts the data in the datastore. I like this because at the end of the process I can do a /data/get and I have a complete JSON variable. I use this same variable to pass into my local web page an parse with json/AJAX and I also post to my data server out on the web if it is time to do so.
Everything was OK until I started to reach a certain number of puts/gets
- The puts on the python very slow (I believe this is due to the opening/closing of put command for every key
-- I have tried everything to speed this up (JSONTCPclient, bridgeclient, etc) - With the large JSON packet the web site would receive just pieces and then eventually stop taking in new values
-- Don't know why
I believe I can use the bridge side of things just to handle the couple I/O pieces that I may need latter on.
So, I was hoping someone could provide me a little direction on how to store item in memory that can be used for both the python program and the web page.
Any help is appreciated, I am attaching chunklets of code to show what I have tried so far.
This is the free running python script
#!/usr/bin/python
import time
import serial
import crcmod
import binascii
import sys
#bring in the python bridge code
sys.path.insert(0,'/usr/lib/python2.7/bridge/')
from bridgeclient import BridgeClient as bridgeclient
# key:value dictionaries to hold all data for web page and Arduino side
# Bridge is required to be running for these to work correctly
dictText={} #Human Readable Key:Value
dictTextSlow={}
dictHex={} #Hex String key is the command, value is the response with ID
---- bunch of boring stuff ---
---- Stuff I tried for speeding up the puts -----
#--------------------------------------------------------------------------------
# Function: keyValuePut
# Dictionary: dictText
# this was very fast, issue was that it would cause issues with the local www
# //arduino.locaa/data/get would have incomplete data ALOT
#--------------------------------------------------------------------------------
#from tcp import TCPJSONClient
#json = TCPJSONClient('127.0.0.1',5700)
#def jsonValuePut(dictName):
#json = TCPJSONClient('127.0.0.1',5700)
#for key,value in dictName.iteritems():
#json.send({'command':'put','key':key, 'value':value})
#time.sleep(10)
#json.close()
client = bridgeclient()
client.timeout = 30
#--------------------------------------------------------------------------------
#Function: keyValuePut
#Dictionary: dictText
#Very Slow - opens and close port for each key:value put
#--------------------------------------------------------------------------------
def keyValuePut(dictName):
for key,value in dictName.iteritems():
#print key,value
client.put(key,value)
------------------- This is the usb/rs232 port open function ----------------------
def cfgPort():
global usbSer
usbSer = serial.Serial( port='/dev/ttyUSB0', baudrate=38400, parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE, bytesize=serial.EIGHTBITS, timeout=1 )
if usbSer.isOpen():
print "Port already open"
return True
else:
print "Opening Port:"
----------- Main loop ------------------------------------
the getXXX function parse the received data and put it into a dict in key:value format
the
while 1:
print '-------- Configuration -----------------'
counter = 0
#getTotals()
#time.sleep(.250)
#getErrorLog()
#print '--- Writing Slow Text Dictionary ----'
#keyValuePut(dictTextSlow)
#jsonValuePut(dictTextSlow)
#clientValuePut(dictTextSlow)
while counter < 30:
print '--------- Scanned data ---- Counter: %s -------------' % counter
getTimeDate()
time.sleep(.250)
getSwVersion()
time.sleep(.250)
getConfig()
time.sleep(.250)
getStat()
time.sleep(.250)
getStates()
time.sleep(.250)
print '--- Writing Fast Text Dictionary ----'
#keyValuePut(dictHex)
#print dictHex
keyValuePut(dictText)
#jsonValuePut(dictText)
#clientValuePut(dictText)
print dictText
counter +=1
So that is the continuously running script on the linino side, this is the js that gets the data and updates the index.html with received json package - It is based off of the keystore_manager_example on the Yun itself
/***************************************************************************
This is what runs on a page refresh/load
On Clicks are handled here also
***************************************************************************/
$(function() {
$("#storage").html(please_wait_message);
storage_retrieve_all(repaint_storage);
function storage_retrieve_all(callback) {
var ajax_call_options = {
url: "/data/get",
timeout: 10000,
success: callback,
error: error_handler
};
add_basic_auth_headers_if_password(ajax_call_options, $("#password").val());
$.ajax(ajax_call_options)
}
function repaint_storage(storage) {
$("#storage").empty();
if (storage.value.length === 0) {
$("#storage").html("Storage is empty!");
return;
}
NumA = storage.value["x"];
NumB = storage.value["y"];
/*
//Use this to print out all the keys and their values
for (var key in storage.value) {
html += "<li>" + key + ":" + storage.value[key] + "</li>";
}
//document.getElementById(put id here).innerHTML = storage.value[put key here];
document.getElementById("vince1").innerHTML = html;
*/
document.getElementById("a").innerHTML = storage.value["A"];
document.getElementById("b").innerHTML = storage.value["B"];
document.getElementById("c").innerHTML = storage.value["C"];
document.getElementById("d").innerHTML = storage.value["D"];
document.getElementById("e").innerHTML = storage.value["E"];
document.getElementById("f").innerHTML = storage.value["F"];
document.getElementById("g").innerHTML = storage.value["G"];
document.getElementById("h").innerHTML = storage.value["H"];
document.getElementById("i").innerHTML = storage.value["I"];
There must be an easier and more reliable way to do this!! I just can't seem to figure it out.
