Hi,
I believe that it is the norm. I'm experiencing similar upload times. The new thing about the Due vs the Uno or the Mega is that it needs to "flash" the memory, and writing and verifying the pages seems to be the bottleneck.
I've tried to disable the "verify code after upload" in the preferences however it does not seem to work as code is still verified.