The Fritzing diagram is, as usual, most uninterpretable but is also clearly wrong. The radio is probably drawing current the entire time.
Get rid of all those parts and put the radio to sleep as well, as demonstrated in this excellent tutorial: Gammon Forum : Electronics : Microprocessors : Solar powered Arduino
I read that if you drop the voltage without the regulator - the arduino will slow down to 1Mhz
You may have read that, but it is utter nonsense.