As I understand it, only electrolytic capacitors are marked with polarity, right? Most larger capacitors (1uF, 10uF ++) can only be found in electrolytic style.

I what kind of circuits is it NOT A GOOD THING (yes, I've smelled a burned capacitor) to use electrolytic capacitors? Or vice versa if that scenario exists.

Any rule of thumb or is this a non-issue?

Tantalum Capacitors are polarized as well.

You can get both polarized and non-polarized electrolytic capacitors in axial and radial lead configurations. In the non-polarized version you also have a choice of common negative or common positive since they are two capacitors with like poles tied together.

Handy guide but by no means exhaustive