what is the best protocol to connect more than 512 devices to an arduino??

why not using wireless? No problem with the load!
XBEE to the rescue ! - 16 (or even 64) bit addressing mode should do it

Question is which bandwidth do you need?
How often the positions change per second per steppenmoter