That's actually pretty sad. Increasing memory size and maintaining transfer speed is never going to work...
My sketch is 13 KB on my Arduino Mega and 80 KB on my Due, so I guess every medium sized sketch will take half a minute to upload.
If anyone has an idea, please post it here 