Henradrie:
...but honestly, the circuit is about as simple as it gets...
And yet you speak of Servos, and a Due...yet your "schematic" includes none of this. Also, no mention of the Potentiometer's resistance. All things that would help to know.
Have you read: How To Use This Forum?