Go Down

Topic: is possible? heterogeneous common bus input, common bus output protocol (Read 3560 times) previous topic - next topic

joaquin0

Sorry that every now and then I these strange questions about communication.

TLDR:

- I need to make a network of arduinos that will receive data, process it and send it to other output arduinos.
- The destination and origin of the messages is given by wether they are plugged.
- They must allow many inputs and outputs


I have been studying and struggling on how to make a kind of distributed processing environment using arduinos, and of course, the communication protocol is the hardest part. I tried to make my own protocol with a simpler scheme, but I learned that I would need a lot of low level knowledge on AVR and C programming that I don't have. For the sake of prototype I am thinking to use ready made libraries, but it is not so easy yet:

I describe below what I am trying to do in terms of communication. But an image will make it way easier:

[see attached image]


I mainly need each arduino to have n possible data inputs, and n possible data outputs. The best would be that n is 'virtually unlimited', but it will be ok for now if n is just more than 1, in which case I could do a module that forks the stream, effectively making a virtually unlimited amount of outputs (at cost of speed), but that is not well desired.

*The data outputs can all be the same, as the output value will always be the same to all the "children". However, some "children" may be busy, meaning that perhaps it is easier to keep outputs independent.

The devices are dinamically plugged and unplugged, but they can be already powered then, avoiding hot plugging problems (thanks, grumpy_mike http://forum.arduino.cc/index.php?topic=476989.msg3258658#msg3258658 )

I also need the "network" to be "heterogeneous", meaning that hopefully it doesn't need one single master, specially because there are many isolated communication nodes. If I am a node, I know what I am sending to my children node, but I have no idea of what that "children" node is sending to my "grandchildren" node.

If interested, an example application is the one in this video (https://www.youtube.com/watch?v=YdsDUqan77s), only that in hardware, with all the changes that it implies to the idea.

So, a thought solution was that each device (arduino) would use the I2C protocol twice. Each device is both a slave and a master: it is a master to all the devices down, and a slave to all the devices up. Would this work? or will I be wasting my time, as I have been quite some days already?
I know that mega readily does that, but I think it is on the expensive side. Whilst I will try to avoid it, if it is too much of a hassle, I can go for it.
if so, what is the best solution?
Am I dreaming of making a rocket to mars?



[note: I posted the same question in the AVR freaks forum http://www.avrfreaks.net/forum/heterogeneous-common-bus-input-common-bus-output-protocol]

DrDiettrich

If you want to send message from and to arbitrary nodes of your network, every node must have a unique address. Then you can connect all nodes to a singe I2C, Ethernet or any other bus.

Otherwise you need hardware that can listen to a number of channels at the same time, like the four Serial channels of a Mega. Or each node must supply signals indicating ready-to-send/receive, so that a next-level controller can serve the individual channels.

johnwasser

Did you have some particular form of 'data' in mind?  On/Off signal?  Fixed-width pulses?  Variable width pulses? Analog voltage?  Asynchronous serial data?  Synchronous serial data?
Send Bitcoin tips to: 1G2qoGwMRXx8az71DVP1E81jShxtbSh5Hp

Paul_KD7HB

In my personal experience, your biggest challenge will be in "error control and recovery". If you have determined a message from any device is corrupted and needs to be resent, how does that work. In same vein, if you send a message to some anonymous device, how will you know it actually exists and received the message as you sent it?

Paul

MorganS

Designing a network from scratch is absurdly hard. The number of things that you have to think about is almost unimaginable. It's as hard to do as outer space is big.

Say the master calls A. A responds. But there's a glitch on the line and the master doesn't hear all of A's response. The master is expecting a response of a certain size so it waits for A to finish. A thinks it has already finished and doesn't have anything more to send. The master isn't calling A because it already got some of the response. Meanwhile B has a really urgent message to send but it knows the master is talking to A and it isn't allowed to say anything while A has the control of the line. But it saw the complete response from A and knows that the line is now vacant so it is allowed to send its urgent message. The master hears this but it thinks it's valid data coming from A, not B's emergency signal so then it has a long message that it thinks is from A and it must have some method of checking that this is really what A meant to say and maybe it has to ask A to repeat the response, except B is still trying to get a response to its emergency message...

And that's not even the most complex thing that can go wrong!

But all this hard work has been done for you, many times even. There's a lot of different network protocols out there which have been thoroughly planned to cover every possible sequence of events. How do you choose?

Start with the distance. How far do you need to communicate? These days, radio transmitters cost less than 20 feet of wire, so radio is a very cheap way to do it. (Look at WiFi, nobody plugs their laptops into the network any more.) I2C is great but distances are very limited.

Then look at the expected interference. Are you communicating between devices in a forest or in an industrial plant? (Animal interference is a nasty one. Rats eat cables.)

At some point, you will need to consider practical concerns like how do you power these devices and if they need waterproof plugs or something. That can affect which network you choose.

Do you need true peer-to-peer networking or can you have a master which addresses the slaves individually? For Arduino-type projects, master-slave is significantly easier.

Do you need the slaves to be "smart" where they know what kind of sensor they are and can describe that to the master or does the master know that slave 1 is always the humidity sensor? (Smart is very hard to do. Do that later as a secondary layer on the network. It shouldn't be a service provided by the basic communication layer.)
"The problem is in the code you didn't post."

joaquin0

Did you have some particular form of 'data' in mind?  On/Off signal?  Fixed-width pulses?  Variable width pulses? Analog voltage?  Asynchronous serial data?  Synchronous serial data?
Sorry that I missed that important detail. I plan to send short words. Perhaps three or four bytes, at a high rate.

joaquin0

all this hard work has been done for you, many times even. There's a lot of different network protocols out there which have been thoroughly planned to cover every possible sequence of events. How do you choose?
In my personal experience, your biggest challenge will be in "error control and recovery". If you have determined a message from any device is corrupted and needs to be resent, how does that work.
So yes, my constrain is on not having too many lines, and being able to unplug. I don't plan to cover distances, and I don't need it to be 100% error proof, but just fast. I don't even need acknowledgement. I totally realize that I should use some other protocol.  So that's why I thought that I2C was a good idea; (except that it has ack) but in this case, I would need to make each node a master downstream, and a slave upstream. Do you think it is possible?

srnet

Sorry that I missed that important detail. I plan to send short words. Perhaps three or four bytes, at a high rate.
And you also missed the important detail of what you mean by 'high rate'
http://www.50dollarsat.info/
http://www.loratracker.uk/

joaquin0

And you also missed the important detail of what you mean by 'high rate'
I have worked a lot with MIDI. Midi tends to get a bit too much latency/ clogging too easily, but is about to be fast enough. I don't know enough as to tell a number confidently.

Robin2

I would not be quite as pessimistic as @MorganS in Reply #4. Yes, all those things need to be dealt with but I reckon they could be with a fairly simple system that uses a fixed message length.

What I am struggling with, when I look at the diagram in the Original Post, is how the devices can be connected together. It is all very well to join them with squiggly lines in a diagram on a page - but the real world needs something a bit more substantial.

Are you aware of the Mesh library for nRF24L01+ wireless transceivers. I have not used it but it seems to have the characteristics you are looking for.

...R
Two or three hours spent thinking and reading documentation solves most programming problems.

DrDiettrich

I2C can do what you want. It allows for multiple masters and collision handling (bus arbitration). The achievable speed depends on the extent (cable length) of the network and number of collisions, and the number of nodes is restricted to 120 (7 bit addresses). These restrictions can be reduced by hardware and firmware, that is not included in the Arduino hardware (multiplexers, line drivers...).

All you need for an exploration are multiple Arduino boards (or other I2C devices), which you'll need anyhow. Find out more yourself...

joaquin0

What I am struggling with, when I look at the diagram in the Original Post, is how the devices can be connected together. It is all very well to join them with squiggly lines in a diagram on a page - but the real world needs something a bit more substantial.

Are you aware of the Mesh library for nRF24L01+ wireless transceivers. I have not used it but it seems to have the characteristics you are looking for.

...R
Yep. Let's imagine for now that they will be RCA cables. I have not designed that yet, because the protocol may require different wiring and so. So the idea is that I can provide the object with many data outputs in the simplest possible way (i.e. just adding RCA sockets). And yes, data length can be fixed.

I was not aware of the RF24 protocol or the library whatsoever. I like that it is designed to connect and disconnect automatically from the devices. The problem is that it works with a single network, whereas I need each network to be isolated; meaning that one module needs not to be aware of what other modules are sending, only if they are physically plugged. If I were to use RF24, I would need to change the concept of what I am doing, in which case, the I2C library would also be useful.

joaquin0

I2C can do what you want. It allows for multiple masters and collision handling (bus arbitration).
Yes, but I2C leaves me with two questions:

- can an arduino handle two I2C networks? because in such a case, it would need to have one input network in which he is a slave, and one output network, in which he is a master.

- can a node be removed without blocking the net or causing problems? I understand that I2C needs ACK once a message has been sent. Removing an element would result in a node going to timeout repeatedly, unless there is a broadcast function that would not require ACK from the slave nodes.

perhaps if all modules were to share one same I2C network, then what you would plug is just a cable that lets one module to "poke" the other module and tell it that the message is for him, as a way for elements to let know that they are plugged together. The cable patching becomes some sort of illusion trick, where the user expects data to pass through the cable, but actually data goes through the common bus. I find this solution a bit unelegant though.

DrDiettrich

Only few controllers (Due?) have two I2C buses.

Every single byte is acknowledged, in hardware. If not, the transmission terminates with an error. A newly plugged node can broadcast a message, telling its address to all other nodes.

joaquin0

It would sort me out if there was a cheap serial buffer chip. Something that I can have one per input. And once any serial message arrives, it stores it until the arduino has time to read it. If so, it would be even possible to put a multiplexor, and read these one by one. Does such a thing exist? I have been searching, but so far have found nothing.

DrDiettrich
Only few controllers (Due?) have two I2C buses.

Every single byte is acknowledged, in hardware. If not, the transmission terminates with an error. A newly plugged node can broadcast a message, telling its address to all other nodes.
I thought that you could make a serial bus out from a couple of digital pins, in a SoftwareSerial fashion. Is it not?

Go Up