I am in need in developing a module that requires two physical ethernet ports to be available so that data can pass through one port (be seen via serial output) and pass through the other output to an end device. This device will not be filtering standard HTTP requests, but instead be sifting through raw byte data, so nothing to 'heavy' so-to-speak.
I have the standard Arduino uno with just one Ethernet shield and I notice that the shield has ICSP pins which can only connect to the arduino board, so if I piggy-back another ethernet shield on top of the existing ethernet shield, I cannot make use of the ICSP pins, is there any way around this or better yet, what role do the ICSP pins play in the ethernet shield? It could be something I may not even need at all.
So in short, I'm just seeing the feasibility of this.
I'm pretty good with one ethernet shield, and I wouldn't try that. Connecting the shield will be the easy part compared to modifying the ethernet library to handle both devices.
The newer shield use the ICSP pins for the SPI data lines, and D10 is the w5100 slave select. You must remove and resolder a new 6 pin connector with the longer pins to accommodate the second shield. You must use another pin for the second w5100 slave select.
Both shields and the Uno may require more power than the voltage regulator or usb can handle.
Now you must modify the ethernet library to use both slave selects.
Connecting would be the easy part, and yes I would need to repin the ICSP port if it IS required...
but couldn't someone just call to two ports to open when firing up the setup() function?
That is the right question. Yes, you can. I have server code that monitors two ports with different web pages on each. Is that what you are looking to do?
SurferTim:
That is the right question. Yes, you can. I have server code that monitors two ports with different web pages on each. Is that what you are looking to do?
Yes basically, but the next challenge is to actually piggy-back the two shields, is that possible? or will I run into a hardware wall if the arduino is requesting resources from either shield?
I'll require both shields so that I can enable the two physical ports.
PeterH:
Can you explain again why you need TWO shields? I'm not seeing any need from what you've said so far.
A pass-through like a gateway or switch.
The device will have an eth0 and and eth1
the eth0 ethernet port will receive packet data via interception (like WireShark) from the preparatory device and display the information through the Arduino's TTL serial display.
The eth1will pass that data out of the Arduino to another target device for its final manipulations.
This is not your classic server/client relationship and does not follow standard HTTP protocol.
I have attached an image showing the setup, you will see the data flow and the two devices in question
What data rate do you need? This is going to be like hooking up a fire hose with a drinking straw in the middle. It sounds like you're trying to make a network tap. An Arduino is a really terrible way to achieve that, I don't even think it could keep up with the "idle" chatter caused by computers just being plugged into the network, never mind actually sending/receiving data.
The cheapest way would be with a laptop running wireshark, and a network tap. The second cheapest would be a dedicated PC with two or more NICs, running PFSense.
Yeah, but I'm still not seeing why you need the input and output to be in separate collision domains. Gateways and switches are very different things and it's not at all clear that this is either of those things. What is this device actually doing?
PeterH:
Yeah, but I'm still not seeing why you need the input and output to be in separate collision domains. Gateways and switches are very different things and it's not at all clear that this is either of those things. What is this device actually doing?
How could it not be more clear? you have an input and an output, stuff goes in, stuff comes out.
It's reading RAW byte data from one device and sends it to another, that raw data is a specific, in-house protocol.
I want the Arduino to just SEE what's being passed through it, it will in no way alter the data.
wizdum:
What data rate do you need? This is going to be like hooking up a fire hose with a drinking straw in the middle. It sounds like you're trying to make a network tap. An Arduino is a really terrible way to achieve that, I don't even think it could keep up with the "idle" chatter caused by computers just being plugged into the network, never mind actually sending/receiving data.
The cheapest way would be with a laptop running wireshark, and a network tap. The second cheapest would be a dedicated PC with two or more NICs, running PFSense.
Data rate can be anything really, the requests being sent through that line are occurring once a second, for example it's literally "saying" this from one device to the other:
"Hey, I'm alive!"
"Okay sending a byte of data"
"Hey, I'm alive!"
"Okay, sending another byte of data"
Now these bytes of data are accumulated / created at the host device and they relate to changes in the environment.
If the whole '2 port arduino' isn't a good idea or just not implementable at all, would a switch do the same as long the devices are able to broadcast on all ports of a switch?
For example, if the host device is on port 1 of 4 on a switch, it broadcasts its message to all 4 ports of the switch? If that's the case I can plug in the Arduino to port 2 and the end device on port 3.
RET80:
How could it not be more clear? you have an input and an output, stuff goes in, stuff comes out.
It's reading RAW byte data from one device and sends it to another, that raw data is a specific, in-house protocol.
I want the Arduino to just SEE what's being passed through it, it will in no way alter the data.
It's not clear because nothing you've said so far precludes having the output interface and the input interface be the same network interface. Ethernet is not like a serial interface, it can carry multiple protocols in multiple directions with multiple senders and receivers all connected to the same network segment. Unless you have some reason why you need to prevent the input packets from being visible on the output network - which you haven't hinted at so far - I can't see why you insist that the input and output networks need to be different. Maybe there is a valid reason, but you haven't given it yet which is why I'm asking the question. If this is a false assumption on your part, you're making this problem a lot harder than it needs to be.
RET80:
How could it not be more clear? you have an input and an output, stuff goes in, stuff comes out.
It's reading RAW byte data from one device and sends it to another, that raw data is a specific, in-house protocol.
I want the Arduino to just SEE what's being passed through it, it will in no way alter the data.
It's not clear because nothing you've said so far precludes having the output interface and the input interface be the same network interface. Ethernet is not like a serial interface, it can carry multiple protocols in multiple directions with multiple senders and receivers all connected to the same network segment. Unless you have some reason why you need to prevent the input packets from being visible on the output network - which you haven't hinted at so far - I can't see why you insist that the input and output networks need to be different. Maybe there is a valid reason, but you haven't given it yet which is why I'm asking the question.
Well then, how would you proceed with this setup?
As you can see the data flow is only in one direction from origin to end device with the Arduino being the 'patch' in the center.
The input and output packets are both visible. Imagine if you will (with the diagram) if the Arduino was removed, it would be a simple, direct connection from host to end device as it sends data unilaterally in one direction. All the Arduino will do is just be a transparent patch between the two devices to sniff data.
I'm a network admin at work. I play with Arduino in my off-hours.
This project is technically possible. Perhaps not with two stacked shields, but it should be possible with with one shield using the standard library and a bunch of wires from unused pins to a second shield using a hacked copy of the library to keep the two separate in software.
That said, I believe I saw someone post on this forum that 14KB/s was the best Ethernet throughput they could get on an Arduino. That's about 0.1Mbps -- compare that to a typical 100Mbps Ethernet connection. If you're copying your data out a second NIC, cut that throughput in half. Then subtract from that the time you'll spend examining the packets to pull out whatever data you're looking for.
If you want a small, power-efficient packet sniffer, a Raspberry Pi with it's built-in NIC plus a USB NIC is going to be literally orders of magnitude faster.
Is the output actually going to be different to the input? You refer to sniffing, not intercepting. If the output is unchanged from the input then this rules out all the possible reasons I can think of for the input and output to need to be separate.
PeterH:
What protocols are being used to convey the data?
Is the output actually going to be different to the input? You refer to sniffing, not intercepting. If the output is unchanged from the input then this rules out all the possible reasons I can think of for the input and output to need to be separate.
I cannot give those protocols out, I am part of an embedded company and what we use is proprietary to the company, what I can say is that it is UDP-like.
The output and input are going to be the same, completely and totally the same, the data will not be changed or modified in any way.
If the input and output are the same, how can the data be outputted to the end device without the second ethernet port? It cannot just magically appear on the end device, it has to travel from end to end, hence the two ethernet ports, the device acts as if it's a patch from end to end.
Now that I think of it...
I think there would be no need to have ethernet shields at all, just have a physical patch with two jacks that pass from end to end and have the Arduino tap into TX and RX lines of the two physical jacks to read raw data, so there would be no need for the ethernet shields, just two jacks with a T-split on the TX and RX lines of the ethernet cable.
RET80:
I think there would be no need to have ethernet shields at all, just have a physical patch with two jacks that pass from end to end and have the Arduino tap into TX and RX lines of the two physical jacks to read raw data, so there would be no need for the ethernet shields, just two jacks with a T-split on the TX and RX lines of the ethernet cable.
A 16MHz Arduino has little hope of reading two 100MHz signals.
RET80:
I think there would be no need to have ethernet shields at all, just have a physical patch with two jacks that pass from end to end and have the Arduino tap into TX and RX lines of the two physical jacks to read raw data, so there would be no need for the ethernet shields, just two jacks with a T-split on the TX and RX lines of the ethernet cable.
A 16MHz Arduino has little hope of reading two 100MHz signals.
Responses from the source device occur every 1,000ms
It sounds like you're trying to make a network tap. An Arduino is a really terrible way to achieve that, I don't even think it could keep up with the "idle" chatter caused by computers just being plugged into the network, never mind actually sending/receiving data.