You have got to ask yourself how it knows not to send those packets when the destination isn't there. I'm guessing that ARP for the destination IP fails, since there is no gateway in the configuration, in other words the connection is direct.
One way to test this hypothesis would be to change the sketch to configure a default gateway to route the IP traffic through -- there must be a router on the network, yes? Packets intended for the other arduino would reliably get to the gateway, unburdening the sending arduino. Then they would be dropped by the router if it can't deliver them, which I'm guessing is better than the present behavior, and maybe even what you want.
Or I could be barking up the wrong tree entirely. But it's an easy test.
-br