a magnet which can be turned off (but doesn't require power to stay on)

You can find reverse EM's

Some are "through EM's" which means they can be used in two directions as a "shuttle"

Some also have spring returns which can in some cases be used to keep the pole in a set position.

Some even have dual poles with a centre position at zero voltage.

I would suggest you look a little further into "electro magnet arduino" on google or such.

PieterP's suggestion is also an excellent one.