For the panning action, just because your axis is not where the center of gravity is, doesn't change anything meaningful. The difference is simply dealt with in the bearings or whatever friction-reducing system you go for.
E.g., a lazy susan can be a good deal off balance weight wise and the movement or interest, being on a perpendicular vertex, is theoretically unchanged (realistically trivially affected by friction as mentioned).