Presumably the number is sent more than once while it is ringing.
Show us a complete list of what appears in the serial monitor.
You need to detect something that shows when a call starts to be received, then do the toggle action only once per call.