Well, the ARM core is totally different from the AVR Arduinos so things that worked on Uno and Mega won't work on the Due. Writing a sketch for the Due is a good exercise in ONLY using the standard Arduino functions and not using unportable tricks like direct port access.
The datasheet for the chip lists lots of cool features like RTC, Ethernet and DAC but they are either somewhat limited (DAC) or totally unusuable on the Due board (Ethernet). Even the 12-bit ADC is badly hampered by the board layout. Lack of EEPROM is a problem for many people too.
I've yet to see an Arduino sketch which actually needs 32-bit integers or the extra speed of the Due. There must be sketches out there which usefully use those features but they're not being published or advertised here. The main usefulness seems to be simply the extra program space.
The 3v limit is usually not a problem. Many sensors are 3.3v and anything that is 5v can understand 3.3v logic levels and a voltage divider fixes inputs to the Due.