I just saw some YouTube videos of people blowing up capacitors by putting them on the wrong way.
You could get burned, or some material could fly into your eye. I suppose you should wear safety glasses when you power-up a circuit for the 1st time. I've never heard of anyone being injured by an exploding capacitor, but if you've seen videos it's obvious that it could happen. Once, I did somehow manage to "flick" solder into my eye, or nearly into my eye... I had to peel it from my eyelids where they close-together. It was scary and it smarted but I didn't need medical treatment.
I hear that capacitors are bad for you, and the smoke could cause cancer.
The fumes can be pretty nasty. I had some capacitors in an amplifier blow-up a few years ago, and I had to get out of the house. They weren't backwards, but they were old and they were operating very close to their rated voltage. But, I doubt that exposure to something like that a few times in your life will cause cancer.
I read on one website that RoHS capacitors are safe from causing cancer or other health problems. I am wondering if that's true?
This is mostly related to lead. Traditional solder is made from tin & lead. And, the leads for most components were coated in solder (AKA "tinned"). Now lead has a bad reputation and, you can go RoHS or "lead free". But if the part explodes, I wouldn't count on the fumes being perfectly safe. 
Besides getting electrocuted, are there any dangers or health risks in working with Arduino projects and the components? Are you supposed to wear gloves or anything?
We are generally working with 5V or ~12V, and that's safe. I pretty-much have my fingers on low voltages all day. I believe 48V and below is considered "safe" by the regulating agencies. Anything above that and you have to make sure that connections cannot be exposed to the user/consumer. If you are working with 120 or 240V power line voltages (building you own power supplies, or switching/dimming lights, etc.) then you need to be careful.
Also, if you are working with dangerous voltages, you should also be aware that capacitors store voltage (sometimes for several minutes or more, depending the circuit). It's an old "trick" to charge-up a capacitor to 100V or so, and toss it to someone (without touching the leads yourself).
P.S.
There was a post (maybe yesterday) about trying to make soldering safer - I mentioned that soldering is less dangerous than sports. I've been working in electronics for many years... I've been shocked and I've been burned on soldering irons, but I've never personally seen anyone hurt badly enough to go to the emergency room. The worst shock I've had was from a spark plug on a lawn mower, and the worst burn I've had is from a car radiator.