That does not a good way to measure how much cable is on the winch, as the diameter changes with the amount of cable, and this is not accurately predictable (it depends on how the cable decides to wind itself). You have to use a separate wheel for that. Of course that wheel can also be mounted on the cable in a way it can't be removed easily.
An encoder on the gearbox may let you know the thing is stalled as no pulses come in any more.