The bridge allow storing key/value pairs.
I was wondering what memory is used for this? Is it the mcu or CPU memory?
If CPU, it could be used as extra storage for the mcu.
The data is stored on the Linux side. It is intended primarily for communications between the Linux and Arduino processors, and for HTTP style communications from other computers. It could be used for some storage, but I think it it would be a rather inefficient way to do so.
NewLine:
The bridge allow storing key/value pairs.I was wondering what memory is used for this? Is it the mcu or CPU memory?
If CPU, it could be used as extra storage for the mcu.
NewLine,
As already mention, the memory is used for passing data back and forth between the mcu and the CPU. From what I have read, the memory is on the CPU (linux) side. Yes you can store things there.
One odd thing you could do (in theory) is write something to the mailbox, reboot the mcu, and read from the mailbox to get the old values back.
Relatively speaking it is more efficient to write to mailbox, than open the bridge and write something to a waiting process on the linux side.
FWIW: in theory you could turn this off (the mailbox and bridge), but I have not read that part yet.
Jesse
Yes I am currently a 'heavy' user of the bridge communicate between the 2 processors.
Obviously using the bridge just for storage is less performant, tcomapred to storing it in the the MCU RAM, BUT you have lots more of it.
E.g. someone here wanted to store tweets and ran out of memory, storing it "in the bridge" might be a workaround without actually having to setup a communication between the 2 processors.
use eeprom - I got 2 x 24LC512 for £2 - thats 64K bytes each
jessemonroy650:
One odd thing you could do (in theory) is write something to the mailbox, reboot the mcu, and read from the mailbox to get the old values back.
Just to be clear, that's resetting the 32U4 MCU, not the AR3391 processor. Reset the '3391 and you lose the data, even if the 32U4 didn't reset.
And also to clarify, in case it confuses anybody, the Mailbox bridge functionality is more like a message queue, where the put/get bridge functionality is more like regular variables. I assume the original discussion was the get/put functions. When you say mailbox, are you referring to this, or are you changing to the other function? (Put/get should also survive a 32U4 reboot.)
NewLine:
E.g. someone here wanted to store tweets and ran out of memory, storing it "in the bridge" might be a workaround without actually having to setup a communication between the 2 processors.
Technically, it is still a communications between the two processors, you just wouldn't have to specifically write code on the Linux side, as the required code is already there. But how are you going to put the values to the bridge store in the first place? If you have the 32U4 put them there, you haven't saved any memory because the strings still need to be in the Arduino sketch. All you're doing is adding overhead.
So, you will need to write some Linux side code to put the values in the store in the first place. If you're going to do that, I would think it's easier to write a Python script that takes an index number as a parameter, and prints out the required tweet. Then use the bridge Process class to run it. Or, if the tweets are random, simplify it and just have the Python code randomly select a string - no need to pass parameters that way.
You could do probably do it with put/get, but that would not be a typical use case. I don't know if it's been tested with a large number of large values. I don't know if the storage method scales well. But it's theoretically possible.
"In theory, there is no difference between theory and practice. In practice, there is."
- -- Jan L. A. van de Snepscheut*
ShapeShifter:
But how are you going to put the values to the bridge store in the first place? If you have the 32U4 put them there, you haven't saved any memory because the strings still need to be in the Arduino sketch. All you're doing is adding overhead.
True, handn't thought that far...
ShapeShifter:
Just to be clear, that's resetting the 32U4 MCU, not the AR3391 processor. Reset the '3391 and you lose the data, even if the 32U4 didn't reset.::::SNIP::::
"In theory, there is no difference between theory and practice. In practice, there is."
- -- Jan L. A. van de Snepscheut*
ShapeShifter,
to be clear, I just tried to use Mailbox. It failed. So rather than experiment or "do some hand waving", here is the ariticle/blog post I read (earlier this week) on the subject.
Understanding Arduino Yún's bridge
Jesse
FWIW: run-bridge mention in the article is in /usr/bin
bridge.py is in /usr/lib/python2.7/bridge, of course
The script is below.
Jesse
#!/bin/sh
cd /usr/lib/python2.7/bridge
exec python -u bridge.py 2> /tmp/bridge.py-stderr.log
ShapeShifter:
Just to be clear, that's resetting the 32U4 MCU, not the AR3391 processor. Reset the '3391 and you lose the data, even if the 32U4 didn't reset.::::SNIP::::
"In theory, there is no difference between theory and practice. In practice, there is."
- -- Jan L. A. van de Snepscheut*
ShapeShifter,
I stand corrected. After a search for how "bridge.py" gets started, the best I could come up with is that it is either
- triggerhappy
- or
- hotplug2
So I re-read the article. I Quote:
When the sketch is restarted, the bridge is restarted, and the key/value store is reset
Jesse
jessemonroy650:
to be clear, I just tried to use Mailbox. It failed.
I'm sorry to be so dense, but it's still not clear. Maybe it's because that article you link says it's using the Mailbox class when it's really using the Bridge class to do get() and put(). Mailbox is a different class built on top of the bridge. When you say Mailbox, are you talking about the actual Mailbox class, or the unfortunate usage of the word mailbox in that article? Either way, that article has an interesting sentence:
When the sketch is restarted (on Arduino), the bridge is restarted, and the key/value store is reset.
It would appear that the key/value database is in the RAM of the bridge Python process. When the Arduino side does when it starts the bridge is to first kill of an existing bridge process, should on exist. Then it starts a new one. So I guess that would reset the data store.
Back to Mailbox: I may be sounding more than a bit pedantic, bit I think proper terminology is important to prevent confusion. There are two main data passing mechanisms in the Bridge library.
The first are the put() and get() functions in the Bridge class. The key/value pairs act like named variables. Any side (Arduino sketch, Linux process, or HTTP call) can set or read these values. When read, the value is not modified and can be read again. It will stay at the current value until overwritten.
The second method is the readMessage() and writeMessage() functions of the Mailbox class. These implement more of a message queue. Any of the three sides can write or read messages from the Mailbox class, but once a message is read it is gone. It can't be read again. The big difference from the Bridge is if you write the same message contents twice in a row, the receiver reading the messages will get two copies of the message. With the bridge class, if the same value is put() twice, the receiver will see the first value, but will have no idea that the second write took place. It can only see changes in the value, where the Mailbox will deliver each copy.
Now, some operating systems and development environments may provide a shared communications system that acts like the Bridge library put/get mechanism, and it may call them named shared mailboxes. But in the Yun environment, where there is a specific Mailbox class that has a different mode of operation, it's a disservice to call the Bridge class a Mailbox, like Jan-Piet Mens does.
jessemonroy650:
After a search for how "bridge.py" gets started, the best I could come up with is that it is either
- triggerhappy
- or
- hotplug2
Your message came in while I was typing. What do you mean by triggerhappy or hotplug2?
I was looking at the Bridge code the other day. The startup process I saw (working from memory, I'm on my iPad at the moment and don't have any code in front of me) is that the Arduino side Bridge.begin() code first sends a message to the Python side to cancel the bridge. The Python code then runs ps to find any bridge related process, and kills them. Then the Arduino side sends the command to run /ISR/bin/bridge.py to kick off a new instance (as you have shown.)
I nay have some details wrong, but that's the gist of it, working from memory.
Either way, it's clear the bridge is reset when the sketch resets, and any bridge values are not preserved. Pity, that might have been handy somehow...
ShapeShifter,
sorry I can't answer more of your questions. There is an entire directory of python code that ties things together with sockets. I presume that a socket (or something) reads a TTY port (or some port) and injects that information into the bridge.py. In turn bridge.py has an entire multiplexing system (or so it seems) to handle things coming at it.
Mailbox is syntactic candy, a Class (sketch and python), a process, and a metaphor for the process. It basically works as expected. However, it does store data while it is running. (Code below) I can presume data is stored while bridge.py is running from the code. (The code is clean).
I'm sorry I'm being terse. I working on commenting the boot process and I'm three (3) days behind. (Far below) is the list of (29) scripts, only a few (13) are partially documented. I'm just finishing with S10boot. Which by the way, looks like spaghetti. It's badly documented. I basically have to chase down every nuance in ash.
For instance, I found this nonsense in an ash script:
scan_wifi() {
local cfgfile="$1"
DEVICES=
config_cb() {
::::SNIP::::
}
config_load "${cfgfile:-wireless}"
}
After much reading, I found out config_cb() returns 0 (zero) because it is defined else where, but redefined here, and this version only gets used when it's needed and only if scan_wifi is run first. (It's a hack to try to be Object Oriented) It's ugly & messy; It's someone getting cute and wants to show everyone how smart they are. (i get a headache reading this.)
If you are wondering what it is for. It is for the community at large, but I will also hoping to present it ForwadJS next month.
Jesse
class Mailbox:
def __init__(self):
self.incoming = deque()
self.data_store = { }
def run(self):
json_server.run()
if json_server.available() > 0:
try:
self.ext_command(json_server.read())
except:
pass
def ext_command(self, msg):
if not 'command' in msg:
return
command = msg['command']
if command == 'raw':
self.incoming.append(msg['data'])
return
if command == 'get':
if msg.has_key('key'):
k = msg['key']
v = self.data_store_get(k)
json_server.write({ 'response' : 'get', 'key' : k, 'value' : v })
else:
json_server.write({ 'response' : 'get', 'value' : self.data_store })
return
if command == 'put':
k = msg['key']
v = msg['value']
self.data_store_put(k, v)
json_server.write({ 'response' : 'put', 'key' : k, 'value' : v })
return
if command == 'delete':
k = msg['key']
v = self.data_store_get(k)
if v:
self.data_store_delete(k)
json_server.write({ 'response' : 'delete', 'key' : k, 'value' : v })
else:
json_server.write({ 'response' : 'delete', 'key' : k })
return
def data_store_put(self, k, v):
self.data_store[k] = v
def data_store_delete(self, k):
del self.data_store[k]
def data_store_get(self, k):
if k in self.data_store:
return self.data_store[k]
else:
return None
def send(self, data):
json_server.write({ 'request' : 'raw', 'data' : data })
def recv(self):
if len(self.incoming) > 0:
return self.incoming.popleft()
return None
def peek(self):
if len(self.incoming) > 0:
return self.incoming[0]
else:
return None
mailbox = Mailbox()
class SEND_Command:
:::: ad nauseam ::::
S05defconfig
S05luci_fixtime
S09handle_wifi_reset
S10boot
S11sysctl
S11ubus
S18rename-wifi-if-access-point
S19firewall
S20fstab
S20network
S39usb
S48rngd
S49delete_uhttpd_cert
S49generate_new_gpg_key
S50cron
S50dropbear
S50uhttpd
S59luci_dhcp_migrate
S60dbus
S60dnsmasq
S61avahi-daemon
S93triggerhappy
S95done
S96led
S97watchdog
S98sysntpd
S99rngd-turn-off
S99usd
ShapeShifter:
Your message came in while I was typing. What do you mean by triggerhappy or hotplug2?::::SNIP::::
triggerhappy, from what I can tell, is a general purpose event handler -- data arrives somewhere, a port has data, etc.
hotplug2 from all outward appearance handle "hot swapable" hardware -- SD, USB, etc.
What I was looking for, but did not find, was a call to /dev or /proc on the linux side, but again nothing.
"The Longest Yard" is on the tube; the original with Burt Reynolds and Eddie Albert.
Back to the salt mine.
Jesse
jessemonroy650:
ShapeShifter,
sorry I can't answer more of your questions.
I don''t really have any questions. I was simply musing about the potentially interesting concept of the bridge data store retaining data through a 32U4 reboot (which we've disproven) and ranting over the confusion caused by calling Bridge.put() and Bridge.get() a Mailbox mechanism (note my capitalization of Mailbox, while I feel should be reserved for talking about the Mailbox class.) It's confusing to call get/put "mailbox" but I feel it's wrong to call it "Mailbox."
Now, to be clear, I realize that the Linux side implements that side of the actual Bridge.get() and Bridge.put() calls in the mailbox.py file using a class called Mailbox. But I would hazard a guess that more than 94% of the people who write a sketch using the Bridge library will never go looking at the Python code. They will just access it from the Arduino sketch side, and on that side get() and put() are implemented in BridgeClass and not in the Mailbox class. It's unfortunate that such inconsistent naming and design was used on the two sides of the bridge. I guess no matter what you do there will always be confusion. Perhaps I'm being overly critical.
jessemonroy650:
After a search for how "bridge.py" gets started, the best I could come up with is that it is either
I just realized that I got sidetracked by the triggerhappy/hotplug2 question, and didn't address the rest of this. bridge.py is started by the Arduino side code in Bridge.begin() calling it directly. Recall that on the Linux side, the serial port used by the bridge is actually the Linux console port. It is running a CLI.
Arduino\libraries\Bridge\src\Bridge.cpp defines BridgeClass::begin() and contains this code to start the bridge:
// - Ask the bridge to close itself
uint8_t quit_cmd[] = {'X', 'X', 'X', 'X', 'X'};
max_retries = 1;
transfer(quit_cmd, 5);
// Bridge startup:
// - If the bridge is not running starts it safely
stream.print(CTRL_C);
delay(250);
stream.print(F("\n"));
delay(250);
stream.print(F("\n"));
delay(500);
// Wait for OpenWRT message
// "Press enter to activate console"
stream.print(F("run-bridge\n"));
delay(500);
dropAll();
It starts by sending a command to quit a potential already running instance of the bridge. (It's this that causes the datastore to be dumped.) It then sends a ^C to kill any other process that might be running on the port. It then sends a couple carriage returns to help make sure the CLI is running and ready for a prompt. It finally sends the string "run-bridge\n" to cause the CLI to run the run-bridge script that you previously identified. It's basically doing what a human operator might do if connected to the console port and trying to manually start it. There is no magic system component involved to kick it off.
ShapeShifter:
I don''t really have any questions. I was simply musing about the potentially interesting concept of the bridge data store::::SNIP::::
Arduino\libraries\Bridge\src\Bridge.cpp defines BridgeClass::begin() and contains this code to start the bridge:
// - Ask the bridge to close itself
uint8_t quit_cmd[] = {'X', 'X', 'X', 'X', 'X'};
::::SNIP::::
ShapeShifter,
thanks for the code reference.
Back to the salt mine.
Jesse