All that happens is that when I press the button, my serial.print statements stop and the onboard LED 13 turns on (it turns on no matter what pin I use for the LED).
I have also tried removing the serial parts in case this was interfering.
You want attachInterrupt(digitalPinToInterrupt(buttonPin), buttonPin_ISR, CHANGE);
and not attachInterrupt(digitalPinToInterrupt(buttonPin), buttonPin, CHANGE);
One thing you should know is that push buttons tend have contact bounce, so one press ends up being several, a millisecond or so apart. So it probably won't work quite like you expect, but it should be good enough to see if interrupts work. Just don't expect it to perform reliably.
Since the interrupt will be called on both press and release, you can always differentiate between the two within the ISR itself by reading the pin:
Actually since you have the interrupt to be generated on CHANGE you will get an interrupt when you press the button AND release the button. Also be aware that mechanical switches will bounce. Most people choose NOT to use an interrupt for buttons, although I understand this is probably just an exercise for you. You might find it interesting to increment a counter inside the interrupt to see how many times the switch bounces.