How to know what 'type' of caps to use??

Just out of curiosity, what happens, or what could happen, if the regulator asks for one, and I put a regular ceramic in place?

For that matter, what would happen if one replaces a polarized cap for a non-polarized one? I've always followed the regulator's datasheets, so now I'm curious.