This part is wrong.
digitalWrite(relayPin, relayState = !relayState);
It won't change between the two states like you seem to think.
Better be explicit than this fancy "alter a variable in the middle of a function call" stuff.
if (relayState == relayOn)
relayState = relayOff;
else
relayState = relayOn;
digitalWrite(relayPin, (relayState == relayOn) ? HIGH : LOW);