UDP bites the dust after a while.

I presume I am somehow running afoul of the ethernet capabilities of the W5500 Chip, but I've had mixed success depending and how I do it, and what I'm trying to accomplish as to this project in regards to how much I/O and how fast I'm trying to send / recieve the data.

So first off my Source :

#include <ArduinoJson.h>
#include <Ethernet.h>
#include <SPI.h>
#include <Encoder.h>

char packetBuffer[UDP_TX_PACKET_MAX_SIZE];
bool debugLog = true;

// This device is using DHCP if you need to set a static IP, you need to do that here.
byte mac[] = {0x04, 0xE9, 0xE5, 0x05, 0x83, 0xA5};
unsigned short localPort = 8887;
EthernetUDP udp;

// Encoder Setup
// Pin numbers to the pins connected to your encoder.
Encoder X_Encoder(1, 2);
Encoder Y_Encoder(4, 5);
Encoder Z_Encoder(7, 8);

// Initial Encoder Positons.
int cacheX = -999;
int cacheY = -999;
int cacheZ = -999;

// Setup Variables for exposing read json values
bool dataStream;
bool dataReset;

void setup() {
  // Initialize serial port
  Serial.begin(9600);
  while (!Serial) continue;

  // Initialize Ethernet libary
  if (!Ethernet.begin(mac)) {
    Serial.println(F("Failed to initialize Ethernet library"));
    return;
  }  
  Serial.print("Sending from : ");
  Serial.println(Ethernet.localIP());
  // Enable UDP
  udp.begin(localPort);
}

// Get JSON Data
void getUDPJsonData(){
  StaticJsonDocument<100> readDoc;    // Init Json Memory Heap  
  int packetSize = udp.parsePacket();
  if(packetSize)
  {
    //Get UDP Data
    Serial.print("Received packet of size ");
    Serial.println(packetSize);
    Serial.print("From ");
    IPAddress remote = udp.remoteIP();
    for (int i =0; i < 4; i++)
    {
      Serial.print(remote[i], DEC);
      if (i < 3)
      {
        Serial.print(".");
      }
    }
    Serial.print(", port ");
    Serial.println(udp.remotePort());
    
    // read the packet into packetBufffer
    udp.read(packetBuffer,48); 
    Serial.print("Data Read : ");
    Serial.println(packetBuffer);           
    //char json[] = "{\"SendData\": false,\"ResetData\": false}"; // For sizing memory for the decoder.    
    
    DeserializationError error = deserializeJson(readDoc, packetBuffer);
    
    // Test if parsing succeeds.
    if (error) {
      Serial.print(F("deserializeJson() failed: "));
      Serial.println(error.c_str());
      return;
    }
    else {
      Serial.print(F("deserializeJson() succeeded: "));
      Serial.println(packetBuffer);
    }

  
    dataStream = bool (readDoc["SendData"]);
    dataReset = readDoc["ResetData"];
    if (dataStream){
      Serial.println("Told to Stream Data");
    }
    if (dataReset){
      Serial.println("Told to Reset Data");
    }
  }
}

//Reply With Sensor Data
void sendUDPJsonData(){
  StaticJsonDocument<72> writeDoc;    // Init Json Memory Heap  
  
  // Grab Data From Encoder
  int X_Pos, Y_Pos, Z_Pos;
  X_Pos = X_Encoder.read();
  Y_Pos = Y_Encoder.read();
  Z_Pos = Z_Encoder.read();

  // If Streaming is desired
  if (dataStream) {
         
    // If Data has Changed
    if (X_Pos != cacheX || Y_Pos != cacheY || Z_Pos != cacheZ) {
      float mult = .15; // So that each rotation = 360 degrees
            
      // Create the Json Document
      writeDoc["SubjectName"].set("KinoWheels");
      JsonArray values = writeDoc.createNestedArray("Rotation");        
      values.add(X_Pos*mult);
      values.add(Y_Pos*mult);
      values.add(Z_Pos*mult);
      
      if (debugLog){
        // Log
        Serial.print(F("Sending to "));
        Serial.print(udp.remoteIP());
        Serial.print(F(" on port "));
        Serial.println(udp.remotePort());        
        serializeJson(writeDoc, Serial); // Debug String to see what was sent.
      }
      
      // Send UDP packet
      udp.beginPacket(udp.remoteIP(), udp.remotePort());
      serializeJson(writeDoc, udp);
      //udp.println("{\"SendData\": false,\"ResetData\": false}"); // Static test to check for memory leak
      udp.println();
      udp.endPacket();            
    }
    
  }
  // Update position for next comparrison
  cacheX = X_Pos;
  cacheY = Y_Pos;
  cacheZ = Z_Pos;  
  
}

void loop() {  
  getUDPJsonData();
  sendUDPJsonData();     
}

Then the output is what's puzzling... it does most things as I'd expect :

Sending from : 192.168.0.88

Received packet of size 38

From 192.168.0.55, port 8888

Data Read : {"SendData": true,"ResetData": true}

deserializeJson() succeeded: SendData

Told to Stream Data

Told to Reset Data

Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[0,0]}Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[-0.15,0]}Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[0,0]}Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[-0.15,0]}Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[0,0]}Sending to 192.168.0.55 on port 8888

And I keep messing with the encoders when it all of a sudden fails, or I can reset it and run it again, and it fails within 2-3 Lines, or on other times if I go slow and steady I can get 1400-4000 samples, and not run into an issue untill sample 4001 and it's all over... Each time it just locks up.

Now I've been reading some of the other UDP threads, and I understand there's only 24kb on the W5500 chip, so am I running afoul of that, and if so how do I "Fix" it?

I've noticed in the past that that JSON library makes heavy use of String objects. I'd suspect a memory leak and would grab the freemem function or the like to check.

That said, it's odd that it can run for ages or fail almost immediately.

If you can detect memory leaking, the JSON you're using looks pretty simple, you could just parse and create it "by hand".

So I had some issues with the arduino grabbing a DHCP, so I left it be, and then rebooted my pc, I don't know what did it, but it started acting normal again...

So here are the numbers for the tests, I think the memory free test must do something with the memory that was a bit more random, as now it crashes after 1 Sample is being red consistently.

Also here are some of the memory numbers I'm getting... seem fairly consistent up to the crash...

Trying to get an IP address using DHCP

Sending from : 192.168.0.88

Received packet of size 39

From 192.168.0.55, port 8888

Data Read : {"SendData": true,"ResetData": false}

deserializeJson() succeeded: SendData

Told to Stream Data

Free Memory Before JSON: 255224

Free Memory Before Debug: 255224

Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[1,0]}Free Memory After Debug: 255224

The last Free Memory After Debug happens right before the UDP package is getting bundled and sent...
I mean that's an indication that something is overruning the memory there, but I don't know how the UDP library breaks up things or if it does it automatically if it's larger than 1024 bytes.

Serial.print("Free Memory After Debug: ");
      Serial.println(freeMemory());
     
      // Send UDP packet
      udp.beginPacket(udp.remoteIP(), udp.remotePort());
      serializeJson(writeDoc, udp);
      //udp.println("{\"SendData\": false,\"ResetData\": false}"); // Static test to check for memory leak
      udp.println();
      udp.endPacket();     

      Serial.print("Free Memory After UDP: ");
      Serial.println(freeMemory());

I also tried to set the sending "doc", >writeDoc< to be a DynamicJsonDocument, to give it some room to breathe in case the numers / number of significant digits were to increase above a certain length value.
No Change.... except I have more free memory somehow.

Trying to get an IP address using DHCP

Sending from : 192.168.0.88

Received packet of size 39

From 192.168.0.55, port 8888

Data Read : {"SendData": true,"ResetData": false}

deserializeJson() succeeded: SendData

Told to Stream Data

Free Memory Before JSON: 249760

Free Memory Before Debug: 249760

Sending to 192.168.0.55 on port 8888

{"SubjectName":"KinoWheels","Rotation":[1,0,0]}Free Memory After Debug: 249760

So it HAS to be in these lines :

// Send UDP packet
      udp.beginPacket(udp.remoteIP(), udp.remotePort());
      serializeJson(writeDoc, udp);
      //udp.println("{\"SendData\": false,\"ResetData\": false}"); // Static test to check for memory leak
      udp.println();
      udp.endPacket();

When I hard code :

This Crashes>> udp.println("{"SendData": false,"ResetData": false}"); // Static test to check for memory leak

This Works>> udp.println("{"A":10}");

So it has to be something with how long the string is, but you can't tell me udp can only send a couple of bytes ?

Well, you've clearly got plenty of memory - what are you running this on?

UDP packets can be huge in theory, but I recall seeing another thread where someone was having trouble sending more than 191 bytes which led to the conclusion that there was a 255 or so byte buffer somewhere along the way when you consider all the overhead for UDP and ethernet.

You're nowhere near that though, especially with your static test.

I wonder if you have an IP conflict. You say DHCP was acting up, maybe there is another device on your network that thinks it has the 88 address. Try pinging it with the Arduino device turned off.

I Don't feel great about this but in case this helps someone else, I ended up massively expanding the packet buffer to 128 from 24, I don't know if that's too much or too little, but for me it stopped crashing, and runs flawlessly for many minutes, I'll test further after I spin the wheels for a whole day... for now it's just the encoders I have to get the aluminum pieces and put it all together.

#define UDP_TX_PACKET_MAX_SIZE 128

If someone could comment on how dangerous or perfectly normal it is, or what the calculation of that value should be based on the string + buffer, due to changing size of the numbers. I'd appreciate it.

I tried running this on a Mega 2560 At First but I kept running out of interrupt-able pins, due to that 20&21 were being used for i2c for ethernet, and I needed 2 pins/axis for each encoder. Then I got some tips to use pull down resistors, and swap the circuit for those 2 pins, and in the mean time, to get my code right I used a Teensy, although I want this running on a Mega as well, as it's open source and should be adaptable by anybody who wants to mod it. It was just easier to prototype on a teensy, as any pin can be an interrupt, and I had plenty of ram and clock cycles to make this as fast as possible...

But it works now, now to adapt it for Mega + Shield...
UPDATE: In messing with how "Low" can I go, I've gotten it to run reliably with as little as 32.

I'm surprised that the buffer default size was so small. Also that overrunning it caused a crash - I would have expected the library to defend against that. On a Teensy or Mega, I wouldn't worry about giving up the 128 bytes to a bigger buffer, but I would be inclined to examine the library to confirm that there is no bounds checking in case your packets ever exceed the new size.

It's the default "Ethernet Include" so I presume a lot of people are using it, I just found only one article that describes UDP crashes that mentions this issue and suggests that solution, but nobody piped in to say if that was "Normal" I'd also like to know if this is typical to measure the packet size, and then adjust the buffer... I guess I'm worried about unforeseen circumstances... but I've been periodically spinning it and so far... about 24 hrs, no crashes...