Can I power the Due directly via 5V pin?

"5V.This pin outputs a regulated 5V from the regulator on the board. The board can be supplied with power either from the DC power jack (7 - 12V), the USB connector (5V), or the VIN pin of the board (7-12V). Supplying voltage via the 5V or 3.3V pins bypasses the regulator, and can damage your board. We don't advise it."

It might work fine, or it might cause problems with the on-board regulator, difficult to tell from the datasheet.
Do you need the isolation from the main power supply?