Power Arduino DUE board from 3 AA batteries: suggestions ?

Since no one has answered your question, I thought I would give it a shot.
According to the schematics, each USB power line is isolated with A MOS FET.
The voltage used will be the higher of the two voltages. No feedback
should occur.

Hope this helps.