UDP ethernet lag after 15-20minutes

Is it slowing up on the processing end or the Arduino end? Put some Serial.println() calls in there to figure it out.

I user raw data with Linux C with UDP and it does fine.