Besides all good hints about how to get the system slower
(BTW:for me as a "normal human", balancing a 1 ft (30cm) rod is too hard, but I can rather easily keep a 6 ft (2m) rod upright, even in 2 dimensions)
When I studied Control Systems at university long ago, we also had an exercise similar to this one (upright rod balanced in one direction by moving the ground point) Of course the goal at that time was to learn doing the maths, but the real experience was to analyze what data comes out of the sensor(s).
I fear the "30" means "degrees" in your sketch: that warns me that your sensors might not be sensitive enough. When you have an open control loop and keep the rod balanced manually:
- is 0 calibrated properly?
- is the slightest leaning against one finger signaled as != 0 ( and in the correct direction ; ) ?
When you close the loop and lean the rod slightly against one finger, the motor should start immediately and follow your finger.
- with which P does it start swinging ?
- does it still react properly with half of that instable P ?
- does a small D make it act random ( noise is the second problem, I agree with liudr ) or simply instable as with a too big P ?
What you'll need is an automatic 0 calibration ( I )
Hope you can one day add a position control. Good luck!