How to clear the bridge buffer of data?

Hardware Arduino Yún
#1

I have a sketch where on the Arduino side, I pass an array of values over the bridge to the Linux side using:

for loop {       // send array of (playerID, score)
    Bridge.put(key,value);
}

To access these variables from the browser, in my javascript file I use:

 $.get('/data/get', function(response){   // gets everything in the data buffer
        // code to process response object containing (playerID, score) array values
}

I have a momentary push button that I use as a 'clear' button. Once pressed, the global variables in my arduino sketch are cleared, however I'm unsure how to clear the data stored on the bridge. This is causing problems for me as even though I press the clear button, my browser still displays artefacts from the last run-through.

For example:

If I send: (1,2), (3,4), (5,6)
The browser retrieves and displays: (1,2), (3,4), (5,6)

// I then press the clear button

Then I send: (1,7), (5,9)
The browser then retrieves and displays: (1,7), (3,4), (5,9)

I've tried calling Bridge.begin() again, since I read that it has code which is supposed to terminate the bridge and clear out the data. However this didn't work for me and the old data was still seen.

Using client.flush() doesn't help either (it only applies to data that hasn't been read yet by the client side).

I don't want to re-upload my sketch, nor do I want to power cycle etc.

I read that I could use the call:

 /data/delete

I guess I'd run this on the client side? Would I run this as a $.put command?

Sorry about how all over the place this ask is. I'll try my best to clarify anything that's unclear.

#2

fccfcc:
::::SNIP::::

I guess I'd run this on the client side? Would I run this as a $.put command?

Sorry about how all over the place this ask is. I'll try my best to clarify anything that's unclear.

@fccfcc,

I've studied this for several days (a month or so ago), and the best that I can tell is the data is stored in a cache on the Linux side. Which means you canNOT do anything with the sketch directly - other than possibly overwrite your variables.

On your YUN, in /www/cgi-bin you'll find a module luci

root@Arduino:/www/cgi-bin# cat luci 
#!/usr/bin/lua
require "luci.cacheloader"
require "luci.sgi.cgi"
luci.dispatcher.indexcache = "/tmp/luci-indexcache"
luci.sgi.cgi.run()
root@Arduino:/www/cgi-bin#

As you can see, not much there. I got the openwrt team to put up the source code server (luci.subsignal.org) for luci part of the webserver (uhttpd), but luci.cacheloader is NOT there. It might be on the YUN source code server (see below) - which means you'd need to (possibly) download the entire source tree.

Google: openwrt subsignal
http://luci.subsignal.org/

NOW you could delete /tmp/luci-indexcache, but I canNOT tell what will happen. All I know is /tmp/luci-indexcache does not exist until you run a sketch (like Bridge), then when you stop the sketch, the file goes away.

My notes for the source code servers is on the link below.

The Yún CLI Tools
http://codesnippets.altervista.org/documentation/yun/CLI_tools/CLItools.html

Lastly, I'm documenting the Yun, so I'd appreciate know what you find, or knowing what you did.

TIA
Jess

#3

As you discovered, there is no method to delete a key/value pair in the bridge datastore. The best you can do is store a blank value, but this does not delete the key.

What is "(1,2)"? Is that a playerID of 1, and score of 2, or is "(1,2)" the score?

The issue here is that you have a variable number of values. The easy solutions are to store it all in a single variable length key, set all of the existing keys to blank when you clear the data, or have another key which says how many values are currently valid.

In essence, rather than treating your "array" of scores as a variable length array, like you would do in some more modern language with rich container classes, think of your "array" as a simple fixed length C array where you are only using some of the elements at any one time. How would you handle that? Probably by setting unused elements to a value that indicates that they're empty, or by using another variable to indicate how many of the array slots are actually in use.

It will probably be easier to come up with a data model that fits the capabilities of the bridge, otherwise you will have to track down and rewrite the Linux side bridge code to add the delete ability.

#4

jessemonroy650:
I've studied this for several days (a month or so ago), and the best that I can tell is the data is stored in a cache on the Linux side.

And I've now been studying it for a few minutes... :smiling_imp:

Jesse, I see the Python code to respond to the get/put commands from the sketch in /usr/lib/pytyhon2.7/bridge/bridgeclient.py. For example, this is the function that handles a Bridge.put() call from the sketch:

 def put(self, key, value):
    json = self.socket_open()
    json.send({'command': 'put', 'key': key, 'value': value})
    r = self.wait_key(key, json, 10)
    self.socket_close()
    return r

Since you've been studying it for a while, maybe you can answer this question off of the top of your head: the code is opening a socket to the loopback address and port 5700, then sending the actual put command as a JSON request. What is on the other side of the socket handling this request? Is it the luci code you're mentioning? Or is that luci code the code that is handling an incoming web request for the key/value pair? Or does the same code do both?

Looking at the code, I was hoping that the Python stored the data locally in some structure, and the web interface accessed that data, but instead it looks like the Python code passes the data up the chain to be managed by a common handler.

#5

ShapeShifter:
::::SNIP::::
Since you've been studying it for a while, maybe you can answer this question off of the top of your head: the code is opening a socket to the loopback address and port 5700, then sending the actual put command as a JSON request. What is on the other side of the socket handling this request? Is it the luci code you're mentioning? Or is that luci code the code that is handling an incoming web request for the key/value pair? Or does the same code do both?
::::SNIP::::

@ShapeShifter,
Well, I have looked at all the "python" code that is the bridge (bridge.py). I canNOT tell you exactly what it does, as I have not run any test. But just going from what I see

  • There is a python "bridge" server that takes commands in a JSON packets. It can do a variety of things (Run, Read, Write, Wait.) bridge.py uses packet.py to verify the packets and decode them appropriately.
  • Somehow it connects to /ttyATH0 - which is how it talks to the ATmega32U4
  • When it gets a packet, it verifies it (via CRC), then decodes it to put on the TCP socket
  • When it places it on the socket, it writes it to "localhost"
  • At the other end it appears the script /www/cgi-bin/luci catches the data via the cache-loader, but I canNOT tell for sure.
  • From there, when you make a CGI call luci does nothing more than depend on the cache-loader to answer the call.

The one part that is missing is that when writing to localhost implies that you are writing to the Internet (as long as there is something there to catch it.).

However, don't ask me to swear on a stack of bibles beyond that. :wink:

Jesse

#6

jessemonroy650:
Somehow it connects to /ttyATH0 - which is how it talks to the ATmega32U4

I think that part is handled by the Bridge class code that gets compiled into the sketch. Because a getty process (or whatever the OpenWRT equivalent is) is running on ttyATH0, the sketch side code can simply issue a CLI command to start Python and the Linux side code. After that, the Python code just reads from STDIN and STDOUT, which were mapped to the tty when the process started.

As to the rest of what you said... my brain hurts! :fearful:

The one part that is missing is that when writing to localhost implies that you are writing to the Internet (as long as there is something there to catch it.).

Actually, by writing to the loopback address (127.0.0.1) it implies that you are going through the network stack, but it shouldn't actually be going out to the Internet (or even off of the board.) But it is being processed through the network stack, and should be handled by the web server or whatever other server is listening on the port, just as if it had come in over the network interface.

Yes, I agree, that's getting into some fine semantics and picking at details...

#7

ShapeShifter:
::::SNIP::::

Yes, I agree, that's getting into some fine semantics and picking at details...

Yeah, I'm trying not to lose the general audience and break out the Stevenson book (which seemed trivial at the time.) I could get knee deep, but at a certain point in time - vocabulary overwhelms the actual work. :wink:

Jesse

#8

I think a reasonable work-around would be for your "clear" button to trigger a function in your sketch that writes "-1" or some other value that you can easily test for to indicate a cleared data value.

But the general problem intrigues me, so I'm just going to lurk in a dark corner over here and see what you folks discover.

#9

Sorry for getting back so late!

I appreciate the advice and insight everyone!

I still haven't found any particular approach I like, but I am also not very knowledgeable with this kind of stuff.

I did end up looking at the python bridge stuff (bridgeclient.py) after ssh-ing into my Yun, but to be honest, I can't fully trace how everything connects. I saw the methods responsible for getting and putting data, but I don't know where those commands are called from etc.

ShapeShifter:
What is "(1,2)"? Is that a playerID of 1, and score of 2, or is "(1,2)" the score?

Your first guess was correct! (1,2) was supposed to represent a playerID of 1 and a score of 2.

In the end I tried to explore the /data/delete/key command. There's no api call that deletes everything in the buffer, but I don't really need that to be the case for now. I just need the playerIDs and the respective scores to be deleted. So I figured I'd create a loop on the front end that would iterate over 1 - 20 (the game supports a max of 20 players) and run delete commands (i.e. /data/delete/1) to create the players from the data buffer.

So to summarise: once the clear button is pressed, a flag is set to true on the bridge's buffer that is polled on the front end. When the front end javascript sees that the flag is true, it runs a post request with a delete URL within a for loop. The delete URL changes for each iteration of the loop to have a different key value that will be deleted.

I wish they just had a Bridge.delete or Bridge.flush command. That would be nice.

Maybe when I find some more time I will dig into this more and try to understand the connection between everything better. Right now, I'm having a hard time tracing all the files and functions that get called between a bridge command being invoked on the arduino and what happens on the linux side that makes everything change on the front end. I have bits and pieces, but no complete picture.

#10

fccfcc:
::::SNIP::::

Maybe when I find some more time I will dig into this more and try to understand the connection between everything better. Right now, I'm having a hard time tracing all the files and functions that get called between a bridge command being invoked on the arduino and what happens on the linux side that makes everything change on the front end. I have bits and pieces, but no complete picture.

@fccfcc,
I'm the same way. I've been trying to track this down for sometime. If you like, we can share notes.

Jesse

#11

I'm definitely down for sharing notes!

I'll be wrapping up my last semester (hopefully hah), and then I'll probably start exploring the Yun. My biggest gripe when working with it was the lack of documentation. It drove me crazy, and the craze hasn't let go yet. I'd love to save some people headaches if I can, and make it an easier item to work with. Just a quick disclaimer though: I'm quite inexperienced, so some of the stuff I share might not be all too insightful.

#12

fccfcc:
I'm definitely down for sharing notes!

I'll be wrapping up my last semester (hopefully hah), and then I'll probably start exploring the Yun. My biggest gripe when working with it was the lack of documentation. It drove me crazy, and the craze hasn't let go yet. I'd love to save some people headaches if I can, and make it an easier item to work with. Just a quick disclaimer though: I'm quite inexperienced, so some of the stuff I share might not be all too insightful.

@fccfcc,

when mining for gold, the results are always given in ounces recovered per ton of slag. I think it will be good. :wink:

Jesse

#13

Here's how to do it. Use the bridgeclient.py on the Linux side.

In you're Arduino code:

Bridge.begin();
//say you put key,value ("d1","Test")
Bridge.put("d1", "Test");

//call python bridge library on the Linux side to delete the key and the value
proc.runShellCommand("python /somefolder/deleteKeys.py");

in the folder /somefolder on the Linux side, here's the python script you need to create (deleteKeys.py):

#!/usr/bin/python
import sys    
sys.path.insert(0, '/usr/lib/python2.7/bridge/')                                               
from bridgeclient import BridgeClient as bridgeclient

bClient = bridgeclient()                                                                                                          

#put whatever code you want here
#such as cClient.get('d1') and do all the processing with the data 

#then delete key d1
bClient.delete('d1')

bClient.close()
#14