What version of the Due do you have? I've got R3 and the crystal and all parts are populated and work great. The crystal isn't enabled by default, but if you check out my post at Slow Clock External Crystal is Not Used - BUG - Suggestions for the Arduino Project - Arduino Forum it is very easy to turn on.