Arduino Nano strange behavior

Also note that when the int -5 (0xFFFB) is used as a PWM value it gets truncated to a byte (0xFB) and treated as unsigned so the value is 251. You're toggling between a very low brightness (5) and a very high brightness (251).