Firstly you need to try substantial low-pass filtering, perhaps a cutoff frequency of 0.3Hz, and see if that's an improvement.
Beware of high amplitude mechanical oscillation and vibration - ultimately this will overload the sensor and prevent a LPF'd version from being faithful to reality - with significant high-frequency components the analog anti-aliasing filter will need to setup properly if you do your main LPF in the digital domain.
Gyros will certainly give much more accurate information on short timescales, and married with an accelerometer to provide drift-correction you can get good orientation information (except for yaw). So an IMU should definitely work well - if more expensive.