If the error is at all times kept small it is basically linear if the actuators each apply a torque

proportional to input.

I've used the error quaternion to drive 3 PID loops, basically use the atan2(x, r),

atan2 (y, r), atan2 (z, r) as the output drive levels from error quaternion (r, x, y, z)

With your approach you need to integrate and differentiate the quaternion error -

whather that saves CPU cycles compared to 3 real PID loops is an interesting issue.

With large step changes where the error is close to 180 degrees the error quaternion

is under-determined. This means rate-limiting step-changes in the input quaternion

is a good idea.

With a quadcopter for instance there are other issues (Z torque much less than XY

torques, saturation of drive levels causes loss of control due to resultant torque

axis locking up).

Interesting project.