Ethernet UDP is limited to 104 bytes

I’m using a custom arduino-compatible board that has a Wiznet 5500 Ethernet chip on the board. I’m working out UDP communication between the custom board and a PC. The PC can send UDP packets to the custom board that are correctly read. And the custom board can send UDP packets to the PC that are correctly read.

Data transmission works until the data packet that I’m sending increases past 104 bytes. Then only the first 104 bytes are transferred.

PC - normal pc running microsoft windows
Custom - custom board running arduino code

To spell this out:
With a UDP packet that is 100 bytes long:
PC —> Custom : Good!
Custom —> PC : Good!

With a UDP packet that is 200 bytes long:
PC —> Custom : Good!
Custom —> PC : BAD :frowning:

When the packet is bad, the first 104 bytes are correct. Bytes 105 through the end are not transmitted.

So I’ve been digging into the Ethernet library to try to figure out the issue. I’m actually using the Ethernet2 library from adafruit because the chip select pin is hard-coded in the Ethernet library and I needed to change the chip select pin for this custom board. Anyway …

udp.beginPacket(ip, port);
bytesWritten = udp.write((uint8_t *)&data, sizeof(struct data));

In the above code, bytesWritten is set to 104 even when sizeof(struct data) is larger than 104.

So, udp.write() returns 104.

from EthernetUdp2.cpp

virtual size_t write(const uint8_t *buffer, size_t size);

size_t EthernetUDP::write(const uint8_t *buffer, size_t size)
  uint16_t bytes_written = bufferData(_sock, _offset, buffer, size);
  _offset += bytes_written;
  return bytes_written;

Alright, what’s bufferData() doing?

from utility/socket.cpp

uint16_t bufferData(SOCKET s, uint16_t offset, const uint8_t* buf, uint16_t len);

uint16_t bufferData(SOCKET s, uint16_t offset, const uint8_t* buf, uint16_t len)
  uint16_t ret =0;
  if (len > w5500.getTXFreeSize(s))
    ret = w5500.getTXFreeSize(s); // check size not to exceed MAX size.
    ret = len;
  w5500.send_data_processing_offset(s, offset, buf, ret);
  return ret;

OK, so w5500.getTXFreeSize() must be returning 104, but why?

in utility/w5500.cpp

uint16_t getTXFreeSize(SOCKET s);

uint16_t W5500Class::getTXFreeSize(SOCKET s)
    uint16_t val=0, val1=0;
    do {
        val1 = readSnTX_FSR(s);
        if (val1 != 0)
            val = readSnTX_FSR(s);
    while (val != val1);
    return val;

Now we’re getting to the interesting bits. readSnTX_FSR() apparently reads the available memory from the Tx buffer on the W5500 chip. According to the data sheet, the W5500 has 32Kbytes of buffer memory. By default, the Ethernet library assigns 16Kbytes for Rx and 16Kbytes for Tx. There are 8 hardware sockets available on the chip, so each socket is assigned 2Kbytes for Rx and for Tx. So why is this function telling me there’s only 104 bytes available?

It’s also interesting that this function reads the value twice and only sends back a value when two consecutive readings are the same. At first glance that seems like an ugly hack. I wonder what happened when they were writing/testing this library …

Now, where is readSnTX_FSR()?

Ugh, the Ethernet library uses macros to define function names. Can’t even search for the function name. (Personal pet peeve of mine, optimize code for reading, not for writing. A programmer will write code once, but someone will read that code thousands of times.)

in utility/w5500.h
__SOCKET_REGISTER16(SnTX_FSR,   0x0020)        // TX Free Size

#define __SOCKET_REGISTER16(name, address)                   \
  static void write##name(SOCKET _s, uint16_t _data) {       \
    writeSn(_s, address,   _data >> 8);                      \
    writeSn(_s, address+1, _data & 0xFF);                    \
  }                                                          \
  static uint16_t read##name(SOCKET _s) {                    \
    uint16_t res = readSn(_s, address);                      \
    uint16_t res2 = readSn(_s,address + 1);                  \
    res = res << 8;                                          \
    res2 = res2 & 0xFF;                                      \
    res = res | res2;                                        \
    return res;                                              \

Not very helpful, it matches up with the W5500 data sheet.

Does anyone have any tips for how to send UDP packets larger than 104 bytes?

Sorry, I don't know.
Could you add a Serial.println(val) in the getTXFreeSize() function before the return. To be sure.
104 is 13 * 8. Does that ring a bell ?
Could it be a timing problem ?

It's finally working, and I had at least two bugs going on...

The actual struct that wasn't working is 360 bytes long.

uint8_t bytesWritten = 0;
udp->beginPacket(*ip, port);
bytesWritten = udp->write((uint8_t *)msg, size);

Feel free to study this brief section of code and see if you can figure out my horrible mistake. Think about writing more than 256 bytes.

What happens when you stuff the value 360 into a uint8_t? That's right, you get 360 - 256 = 104!! So of course it was working correctly all along. There were times when I was printing the value, and there were times I was experimenting with different size structs. But apparently I didn't try a different size AND print at the same time.

And here's the real kicker. I had a typo on the PC side of things on a sizeof(struct xxxx). Coincidentally, that struct was only 100 bytes. So on the PC side it was only copying 100 bytes from the incoming UDP packet. As I was counting bytes in the Wireshark window, it was all zeros after 100 bytes. The Ethernet chip was telling me it was only sending 104 bytes. I initially assumed that the 4-byte discrepancy was due to me mis-counting bytes on the laptop screen. Ugh.

Well I wasted two days chasing that one down. At least I learned a lot about the Ethernet library!