show us your Arduino code, please. I'll bet there are improvements to be made, but in the last, yes, you'll likely have to throttle your demands. That's a one-lung, 16 MHz cpu, with likely A PC of some form throwing stuff at it.
Sorry, gross assumption there, I'm talking about an Arduino Nano. You might have something different, how about telling us about it?.