Remotely Monitoring Drone Payload Line Length

The thought I had was that the drone may have stability issues with a near 1kg weight swinging around 1000ft below it.

If the final spindle is at the center of gravity, then swinging payload will have minimal affect on the drone stability.

My concern with the rotary encoder is that it could experience mechanical wear over time from being turned so much

The encoders I see have no mechanical wear. The rotating slotted disc is attached directly to the rotating piece (the spindle in your case). It has LEDs and sensors near it but not touching. The existing bearings for your spindle are the only mechanically wearing piece in that system.