Udp.beginPacket() time delay problem

Hi All,

I am new to programming with the Arduino development boards so please bear with me. I am working on a project were I need to implement UDP to communicate between multiple Arduino Mega2560 boards (each with its own Ethernet Shield). One board will be kind of like the Master where it will communicate with the other boards by sending commands and receiving information back from the other boards. I am using an Ethernet switch to connect all the boards together and build a UDP network. Currently I have only 2 Arduino boards that I am experimenting with and I am running into a problem implementing the UDP. I noticed that when I try sending something over UDP by using the following sequence of commands:

Udp.beginPacket(IP_Remote, Port_Remote);
Udp.write("ping");
Udp.endPacket();

If there is nothing on the other end to receive this UDP pocket, then Udp.beginPacket(IP_Remote, Port_Remote) takes a long time to execute (2-5 sec). It also appears that the timing is inconsistent every time I loop through the sequence of commands above. I dug into the EhternetUdp.cpp and Dns.cpp libraries and found that beginPacket() calls getHostByName() which attempts to connect to the remote device multiple times and causes the delay. My first problem is that this timing is inconsistent. I am trying to implement the UDP send code in an interrupt routine that runs every second for n times. This is to check if the status of the remote device. And the second problem is that it takes too long. I would like this to run every 1 second or maybe less. Another thing which I don't understand is why this UDP implementation cares about if there is a remote device on the other end? My understanding of UDP is that it’s connectionless; meaning it just sends information on the network and does not care if the remote device receives that information or not. Another thing I noticed is that beginPacket() always returns 1 whether connection is established or not. Is there a way to check for the remote device before I run Udp.beginPacket()? I really don’t want to have to modify the libraries. Any information is greatly appreciated.

Thanks

#include <SPI.h>         // needed for Arduino versions later than 0018
#include <Ethernet.h>
#include <EthernetUdp.h>         // UDP library from: bjoern@cs.stanford.edu 12/30/2008

byte mac[] = {
  0x90, 0xA2, 0xDA, 0x0D, 0x50, 0x57}; //Ethernet shield MAC Address
IPAddress IP(169, 254, 0, 1);
IPAddress IP_Remote(169, 254, 0, 2);           // IP address for Remote Arduino
unsigned int localPort = 6101;                 // local port to listen on
unsigned int Port_Remote = 6180;        // Remote Arduino remote port


const int LEDPIN = 2;
volatile byte seconds = 0;

// An EthernetUDP instance to let us send and receive packets over UDP
EthernetUDP Udp;
EthernetClient client;


char packetBuffer[UDP_TX_PACKET_MAX_SIZE]; //buffer to hold incoming packet,

void setup()
{
  pinMode(4,OUTPUT); // for the MEGA 2560 to disable SD


  pinMode(LEDPIN, OUTPUT);
  // initialize Timer1
  cli();          // disable global interrupts
  TCCR1A = 0;     // set entire TCCR1A register to 0
  TCCR1B = 0;     // same for TCCR1B

  // set compare match register to desired timer count: (for 1 sec use 15624)
  OCR1A = 46874; // set for 3 seconds but this is too long. Prefer to use 1 second

  // turn on CTC mode:
  TCCR1B |= (1 << WGM12);

  // Set CS10 and CS12 bits for 1024 prescaler:
  TCCR1B |= (1 << CS10);
  TCCR1B |= (1 << CS12);

  // enable timer compare interrupt:
  TIMSK1 |= (1 << OCIE1A);

  // enable global interrupts:
  sei();


  // start the Ethernet and UDP: 
  Ethernet.begin(mac,IP);
  Udp.begin(localPort);

  // start the Serial Port:
  Serial.begin(9600);

}

void loop()
{

  Serial.println("in loop()");

  digitalWrite(4,HIGH); // for the MEGA 2560 to disable SD
  
  // if there is nothing to receive the UDP packet, then Udp.beginPacket() takes too long to executes (2-5 sec.)
  // Udp.beginPacket() always returns 1
  if (Udp.beginPacket(IP_Remote, Port_Remote))
  {
    Serial.println("connected");
    Serial.println(Udp.beginPacket(IP_Remote, Port_Remote));
    Udp.write("ping");
    Udp.endPacket();
  } 
  else 
  {
    Serial.println("connection failed");
  }
  int packetSize = Udp.parsePacket();
  if(packetSize)
  {
    Serial.print("Rcvd packet size of: ");
    Serial.print(packetSize);
    Udp.read(packetBuffer,UDP_TX_PACKET_MAX_SIZE);
    Serial.print(", Contents:");
    Serial.println(packetBuffer);
  }
  delay(50);
}

ISR(TIMER1_COMPA_vect)
{
  // would like to add UDP ping here instead of the main loop
  Serial.println("in ISR()");
  digitalWrite(LEDPIN, !digitalRead(LEDPIN));
  seconds++;
  if (seconds > 10)
  {
    cli();          // disable global interrupts
    TCCR1A = 0;     // set entire TCCR1A register to 0
    TCCR1B = 0;     // same for TCCR1B
    sei();
  }

}

Another thing which I don't understand is why this UDP implementation cares about if there is a remote device on the other end? My understanding of UDP is that it’s connectionless; meaning it just sends information on the network and does not care if the remote device receives that information or not.

It doesn't care, unless it is on the same localnet as the destination. Then it will try for a while to send the packet. If it can't, it drops the packet.

I am trying to implement the UDP send code in an interrupt routine that runs every second for n times.

That is so far from the right way to do it that I can't even begin to tell you how to correct the problems.

You understand, I hope, that UDP is not meant to send and receive responses. It is meant to be used like a radio station. Broadcast a message, If no one hears it, that makes no difference.

If it is heard, but not responded to, no big deal.

If it is heard, and the listener then sends a e-mail to the station saying thank you, that is a bonus.

If you expect a reply, UDP is not the protocol to use. If the timing of the reply is important, UDP is even less appropriate.

You understand, I hope, that UDP is not meant to send and receive responses. It is meant to be used like a radio station. Broadcast a message, If no one hears it, that makes no difference.

Yes I understand how UDP works ;). But the problem is that UDP appears to behave differently based on whether there is someone listens to it or not on the other end. The line of code that contains Udp.beginPacket(IP_Remote, Port_Remote) will execute almost instantly if there is a device to receive the message on the other end but it takes 2 to 5 seconds to execute if there is no device to receive the message on the other end. my idea is to use the interrupt routine to periodically broadcast a message using UDP (just to see if some one is out there) and the main loop would only monitor incoming messages on UDP and do other stuff. I can see where in the library files (EhternetUdp.cpp and Dns.cpp) it attempts 3 times to connect to the remote (when you call Udp.beginPacket(IP_Remote, Port_Remote)) before it gives up but what I don't understand is why it does that?

The line of code that contains Udp.beginPacket(IP_Remote, Port_Remote) will execute almost instantly if there is a device to receive the message on the other end but it takes 2 to 5 seconds to execute if there is no device to receive the message on the other end.

UDP packets are routed just like a TCP packet, except no connection is established between the source and destination. A "connection" is still established between individual devices/routers to pass the packet from router to router along the way. All routers will accept the packet if they are operational, whether the destination device exists or not.

If the router currently handling the packet can route the packet towards the destination, and the next router/device in line takes it, the packet transfer is almost immediate. If the next router/device in line has failed or doesn't exist, the current router will try to pass the packet for a few seconds, then drop the packet. That is it. Nothing more. Nobody is told it dropped.

If the destination device is not on your localnet (different subnet), chances are your gateway router will take the packet immediately with no check to see if the destination device exists. If the gateway router is down, the packet will be dropped after a few seconds, and you would see that short delay there also.

If the destination device is on your localnet (same subnet), the source device will try to pass the packet directly to the destination device, but if it is not there to take it, the packet is dropped after a few seconds.

If the UDP implementation of connect is initiating a network connection then I would conclude that it is faulty. A UDP 'connect' should merely set a default destination address and should not initiate any network traffic. It seems to me that this is the sort of mistake that could easily have been introduced by using the TCP protocol implementation as the basis for a UDP implementation, and not addressing all of the differences.

A UDP 'connect' should merely set a default destination address and should not initiate any network traffic.

How do the NTP packets get to and from the NTP server with no network traffic? They are UDP. Or am I reading that wrong?

I do not believe there is a connection established between the source and destination devices like TCP. The packet is passed from router to router if they are operational, but that does generate network traffic, does it not?

SurferTim:

A UDP 'connect' should merely set a default destination address and should not initiate any network traffic.

How do the NTP packets get to and from the NTP server with no network traffic? They are UDP. Or am I reading that wrong?

UDP is a datagram protocol and TCP is a stream protocol.

With a stream protocol, a connection is established between a client and server by exchanging packets over the network and then content is written over the stream by exchanging additional packets. Hence there are distinct 'connect' and 'send' phases which both involve network traffic.

With a datagram protocol, no connection is established. Hence the 'connect' phase does not involve sending or receiving network traffic, it just updates the internal state of the client to indicate where subsequent datagrams will be sent to. Nothing is sent over the network until you actually send data. Since Udp.beginPacket() function does not have any arguments providing data to be sent, I would not expect that to cause any network activity.

OK. We are on the same page. You are correct. I don't know what would make the Arduino send to the w5100 stall for a few seconds. It seems it should take that packet and return. Unless maybe the next UDP packet send is stalled waiting for the previous one to finish or timeout?

BTW, I wasn't asking that question about NTP packets because I didn't know. Just wondered if you did.

For everyone else reading this, here is the "Question of the Day":
How do the NTP response packets from the NTP server on a public IP get through my router and firewall to my Arduino on a localnet IP?

If you mean why is the incoming UDP packet allowed past the firewalls, the answer is that if it is then it's because the security policy permits it.

If you mean how does it reach the correct IP/UDP address/port, it would require Network Address Translation (and perhaps UDP hole punching or explicit NAT configuration).

If you mean how does it reach the correct IP/UDP address/port, it would require Network Address Translation (and perhaps UDP hole punching or explicit NAT configuration).

I figured you knew. :slight_smile:

Most routers use connection tracking internally for everything the Network Address Translator (NAT) does in that router. It matters not what protocol it used. It "links" the public ip/port to the Arduino private ip/port.

If the connection state never becomes established, which UDP never does, the NAT routing is dropped from the connection table after about 10 seconds on my routers. That is the time the NTP server has to get the response packet back to my router so it will know where to deliver it. After 10 seconds, the packet will be blocked by the firewall.

If the destination device is on your localnet (same subnet), the source device will try to pass the packet directly to the destination device, but if it is not there to take it, the packet is dropped after a few seconds.

So, from what I understand from this discussion is that there is really no way around the delay in Udp.beginPacket() if there is no device to listen to it on the other end (without having to go in and modifying the library files). One thing though is that you would expect Udp.beginPacket() to return 0 if it fails to 'connect' to the receiving device but what I noticed is that it always returns 1. I can see the code in the library files where this is set but logically, shouldn't it return 0 if it fails to 'connect'? The Arduino website documentation indicates that this function does not return which I guess is not correct.

eddy9:
So, from what I understand from this discussion is that there is really no way around the delay in Udp.beginPacket() if there is no device to listen to it on the other end (without having to go in and modifying the library files).

Well, I suppose you could report an apparent bug to the people maintaining the library and wait until they fix it.

You are correct about the return values. The docs are in error. This is from Udp.h in the Arduino cores stuff:

  // Start building up a packet to send to the remote host specific in ip and port
  // Returns 1 if successful, 0 if there was a problem with the supplied IP address or port
  virtual int beginPacket(IPAddress ip, uint16_t port) =0;

  // Start building up a packet to send to the remote host specific in host and port
  // Returns 1 if successful, 0 if there was a problem resolving the hostname or port
  virtual int beginPacket(const char *host, uint16_t port) =0;

  // Finish off this packet and send it
  // Returns 1 if the packet was sent successfully, 0 if there was an error
  virtual int endPacket() =0;

beginPacket returns 1 if there is no problem with the ip or port format, 0 if there is a problem. But does not signify fail or success on the send.

endPacket returns 1 if the packet was sent successfully, or 0 if error.

beginPacket doesn't send anything, it just sets up the packet format. endPacket sends.

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

So this indicates a two way UDP communication with the router?

@zoomkat: Correct. A two-way communication with the first device only. If the destination ip is localnet, the destination ip device directly. If the destination ip is not localnet, the gateway router.

zoomkat:

edit: The endPacket success return value does not indicate the packet got to its destination if the destination is not on your localnet, only whether the gateway router took it or not.

So this indicates a two way UDP communication with the router?

I wouldn't expect so. UDP is an unreliable protocol (meaning that there is no delivery acknowledgment, no explicit error detection/reporting, transmission can fail without the sender being informed). In some situations you might get an ICMP error message but that's not really part of UDP. Does endPacket() wait for ICMP responses? That would be a strange thing for it to do.

I wouldn't expect so. UDP is an unreliable protocol (meaning that there is no delivery acknowledgment, no explicit error detection/reporting, transmission can fail without the sender being informed). In some situations you might get an ICMP error message but that's not really part of UDP. Does endPacket() wait for ICMP responses? That would be a strange thing for it to do.

Would you be so kind as to tell me how UDP packets get to their destination without being routed? It would be a good thing for me to know, being a routing specialist for an ISP.

A two-way communication with the first device only. If the destination ip is localnet, the destination ip device directly. If the destination ip is not localnet, the gateway router.

Why does this require two way communication. I would think the router is coded to localnet non internet routable IP addresses and pass the rest outbound. No need for two way communication.

Did you see my "Question of the Day" above?

How do the NTP response packets (UDP) from the server in Atlanta, or the DNS response packets (UDP) from the server in Fort Walton, get through my router back to my Arduino on a localnet ip, through my firewall and NAT without the routers communicating with each other?

Your router does the same. There is no connection. Unlike TCP, the UDP response packet is not associated with the request packet in any way as far as the enroute routers are concerned. They can't tell a request from a response with UDP. How does it know that particular port on my router's public ip is associated with the UDP port on my Arduino localnet ip to pass that response packet back? Magic pixie dust? Salted peanuts?

I'm not talking about source to destination two way communication like a TCP connection, just router to router as they pass the packet towards the destination ip.

As I said before, if you think this isn't it, please enlighten me on how it works. It would be a good thing for me to know.