Am I having crystal problems?

Do youhave your Pull-Up/Pull-Down resistors wired correctly? Sounds like you have a floating input that tends to stay at its last state.