I'm outputting an XML packet from my duino,
Just how many bytes are you putting in the "packet". How many packets are being generated and sent? I've got a simple little servo control setup using the ethernet shield and the control is very quick. Perhaps you are incurring some XML delay and overhead.