I notice that your code doesn't start the pullup resistors on the switch pins. Do you have pullup resistors wired in? If not your inputs are probably floating which could result in random behaviour.
I'd also recommend using a SPDT switch with a center off position for the signal light. It won't change your sketch but it will put the signal lights on a single switch. Run the the center to ground and each throw to a pin.