Udp.beginPacket() time delay problem

Another thing which I don't understand is why this UDP implementation cares about if there is a remote device on the other end? My understanding of UDP is that it’s connectionless; meaning it just sends information on the network and does not care if the remote device receives that information or not.

It doesn't care, unless it is on the same localnet as the destination. Then it will try for a while to send the packet. If it can't, it drops the packet.

I am trying to implement the UDP send code in an interrupt routine that runs every second for n times.

That is so far from the right way to do it that I can't even begin to tell you how to correct the problems.

You understand, I hope, that UDP is not meant to send and receive responses. It is meant to be used like a radio station. Broadcast a message, If no one hears it, that makes no difference.

If it is heard, but not responded to, no big deal.

If it is heard, and the listener then sends a e-mail to the station saying thank you, that is a bonus.

If you expect a reply, UDP is not the protocol to use. If the timing of the reply is important, UDP is even less appropriate.

You understand, I hope, that UDP is not meant to send and receive responses. It is meant to be used like a radio station. Broadcast a message, If no one hears it, that makes no difference.

Yes I understand how UDP works ;). But the problem is that UDP appears to behave differently based on whether there is someone listens to it or not on the other end. The line of code that contains Udp.beginPacket(IP_Remote, Port_Remote) will execute almost instantly if there is a device to receive the message on the other end but it takes 2 to 5 seconds to execute if there is no device to receive the message on the other end. my idea is to use the interrupt routine to periodically broadcast a message using UDP (just to see if some one is out there) and the main loop would only monitor incoming messages on UDP and do other stuff. I can see where in the library files (EhternetUdp.cpp and Dns.cpp) it attempts 3 times to connect to the remote (when you call Udp.beginPacket(IP_Remote, Port_Remote)) before it gives up but what I don't understand is why it does that?

The line of code that contains Udp.beginPacket(IP_Remote, Port_Remote) will execute almost instantly if there is a device to receive the message on the other end but it takes 2 to 5 seconds to execute if there is no device to receive the message on the other end.

UDP packets are routed just like a TCP packet, except no connection is established between the source and destination. A "connection" is still established between individual devices/routers to pass the packet from router to router along the way. All routers will accept the packet if they are operational, whether the destination device exists or not.

If the router currently handling the packet can route the packet towards the destination, and the next router/device in line takes it, the packet transfer is almost immediate. If the next router/device in line has failed or doesn't exist, the current router will try to pass the packet for a few seconds, then drop the packet. That is it. Nothing more. Nobody is told it dropped.

If the destination device is not on your localnet (different subnet), chances are your gateway router will take the packet immediately with no check to see if the destination device exists. If the gateway router is down, the packet will be dropped after a few seconds, and you would see that short delay there also.

If the destination device is on your localnet (same subnet), the source device will try to pass the packet directly to the destination device, but if it is not there to take it, the packet is dropped after a few seconds.

If the UDP implementation of connect is initiating a network connection then I would conclude that it is faulty. A UDP 'connect' should merely set a default destination address and should not initiate any network traffic. It seems to me that this is the sort of mistake that could easily have been introduced by using the TCP protocol implementation as the basis for a UDP implementation, and not addressing all of the differences.

A UDP 'connect' should merely set a default destination address and should not initiate any network traffic.

How do the NTP packets get to and from the NTP server with no network traffic? They are UDP. Or am I reading that wrong?

I do not believe there is a connection established between the source and destination devices like TCP. The packet is passed from router to router if they are operational, but that does generate network traffic, does it not?

SurferTim:

A UDP 'connect' should merely set a default destination address and should not initiate any network traffic.

How do the NTP packets get to and from the NTP server with no network traffic? They are UDP. Or am I reading that wrong?

UDP is a datagram protocol and TCP is a stream protocol.

With a stream protocol, a connection is established between a client and server by exchanging packets over the network and then content is written over the stream by exchanging additional packets. Hence there are distinct 'connect' and 'send' phases which both involve network traffic.

With a datagram protocol, no connection is established. Hence the 'connect' phase does not involve sending or receiving network traffic, it just updates the internal state of the client to indicate where subsequent datagrams will be sent to. Nothing is sent over the network until you actually send data. Since Udp.beginPacket() function does not have any arguments providing data to be sent, I would not expect that to cause any network activity.

OK. We are on the same page. You are correct. I don't know what would make the Arduino send to the w5100 stall for a few seconds. It seems it should take that packet and return. Unless maybe the next UDP packet send is stalled waiting for the previous one to finish or timeout?

BTW, I wasn't asking that question about NTP packets because I didn't know. Just wondered if you did.

For everyone else reading this, here is the "Question of the Day":
How do the NTP response packets from the NTP server on a public IP get through my router and firewall to my Arduino on a localnet IP?

If you mean why is the incoming UDP packet allowed past the firewalls, the answer is that if it is then it's because the security policy permits it.

If you mean how does it reach the correct IP/UDP address/port, it would require Network Address Translation (and perhaps UDP hole punching or explicit NAT configuration).

If you mean how does it reach the correct IP/UDP address/port, it would require Network Address Translation (and perhaps UDP hole punching or explicit NAT configuration).

I figured you knew. :slight_smile:

Most routers use connection tracking internally for everything the Network Address Translator (NAT) does in that router. It matters not what protocol it used. It "links" the public ip/port to the Arduino private ip/port.

If the connection state never becomes established, which UDP never does, the NAT routing is dropped from the connection table after about 10 seconds on my routers. That is the time the NTP server has to get the response packet back to my router so it will know where to deliver it. After 10 seconds, the packet will be blocked by the firewall.

If the destination device is on your localnet (same subnet), the source device will try to pass the packet directly to the destination device, but if it is not there to take it, the packet is dropped after a few seconds.

So, from what I understand from this discussion is that there is really no way around the delay in Udp.beginPacket() if there is no device to listen to it on the other end (without having to go in and modifying the library files). One thing though is that you would expect Udp.beginPacket() to return 0 if it fails to 'connect' to the receiving device but what I noticed is that it always returns 1. I can see the code in the library files where this is set but logically, shouldn't it return 0 if it fails to 'connect'? The Arduino website documentation indicates that this function does not return which I guess is not correct.

eddy9:
So, from what I understand from this discussion is that there is really no way around the delay in Udp.beginPacket() if there is no device to listen to it on the other end (without having to go in and modifying the library files).

Well, I suppose you could report an apparent bug to the people maintaining the library and wait until they fix it.

You are correct about the return values. The docs are in error. This is from Udp.h in the Arduino cores stuff:

  // Start building up a packet to send to the remote host specific in ip and port
  // Returns 1 if successful, 0 if there was a problem with the supplied IP address or port
  virtual int beginPacket(IPAddress ip, uint16_t port) =0;

  // Start building up a packet to send to the remote host specific in host and port
  // Returns 1 if successful, 0 if there was a problem resolving the hostname or port
  virtual int beginPacket(const char *host, uint16_t port) =0;

  // Finish off this packet and send it
  // Returns 1 if the packet was sent successfully, 0 if there was an error
  virtual int endPacket() =0;

beginPacket returns 1 if there is no problem with the ip or port format, 0 if there is a problem. But does not signify fail or success on the send.

endPacket returns 1 if the packet was sent successfully, or 0 if error.

beginPacket doesn't send anything, it just sets up the packet format. endPacket sends.

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

So this indicates a two way UDP communication with the router?

@zoomkat: Correct. A two-way communication with the first device only. If the destination ip is localnet, the destination ip device directly. If the destination ip is not localnet, the gateway router.

zoomkat:

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

So this indicates a two way UDP communication with the router?

I wouldn't expect so. UDP is an unreliable protocol (meaning that there is no delivery acknowledgment, no explicit error detection/reporting, transmission can fail without the sender being informed). In some situations you might get an ICMP error message but that's not really part of UDP. Does endPacket() wait for ICMP responses? That would be a strange thing for it to do.

I wouldn't expect so. UDP is an unreliable protocol (meaning that there is no delivery acknowledgment, no explicit error detection/reporting, transmission can fail without the sender being informed). In some situations you might get an ICMP error message but that's not really part of UDP. Does endPacket() wait for ICMP responses? That would be a strange thing for it to do.

Would you be so kind as to tell me how UDP packets get to their destination without being routed? It would be a good thing for me to know, being a routing specialist for an ISP.

A two-way communication with the first device only. If the destination ip is localnet, the destination ip device directly. If the destination ip is not localnet, the gateway router.

Why does this require two way communication. I would think the router is coded to localnet non internet routable IP addresses and pass the rest outbound. No need for two way communication.

Did you see my "Question of the Day" above?

How do the NTP response packets (UDP) from the server in Atlanta, or the DNS response packets (UDP) from the server in Fort Walton, get through my router back to my Arduino on a localnet ip, through my firewall and NAT without the routers communicating with each other?

Your router does the same. There is no connection. Unlike TCP, the UDP response packet is not associated with the request packet in any way as far as the enroute routers are concerned. They can't tell a request from a response with UDP. How does it know that particular port on my router's public ip is associated with the UDP port on my Arduino localnet ip to pass that response packet back? Magic pixie dust? Salted peanuts?

I'm not talking about source to destination two way communication like a TCP connection, just router to router as they pass the packet towards the destination ip.

As I said before, if you think this isn't it, please enlighten me on how it works. It would be a good thing for me to know.

How does it know that particular port on my router's public ip is associated with the UDP port on my Arduino localnet ip to pass that response packet back? Magic pixie dust? Salted peanuts?

From my understanding the packet sent by your router has your router's public IP address and specific port contained in it (like a return telephone number and extension), which stays in the packet until it reaches its destination or expires. The destination server may or may not send a response back to your router using the IP address contained in the origional packet. If the return packet reaches your router, the router should be programmed to use the port number origionally sent to identify the LAN originating IP address. No magic pixie dust or salted peanuts needed (at least on this end).