Ok both my memory and my math sucks. The actual number was 4uS and it works out to 250kHz (even if it was 3.3uS it would have worked out to 303kHz not 30,3kHz, my bad ) which is much quicker than what you're getting. The writeup and results are here Evaluating Arduino and Due ADCs.
Before he mods the registers it takes 39uS for the loop which works out to 25,6kHz. Almost spot on what you mentioned, so my suggestion is do this mod and you'll increase your sample rate by a factor of 10. Not 1Msps but certainly a lot closer.
Hope this helps
Andrew