You'd need something to disconnect the capacitor from the battery when you transmit. Capacitors also have a discharge curve opposite from what you get from a battery... The voltage drops quickly at first and then levels-off as approaches zero. Capacitors can be used as batteries if you have very-low current and/or lots of "extra" voltage and an efficient voltage regulator.
With a MOSFET (or transistor, or relay) and some timing in your sketch (look at the LED blinking examples) you can turn-off the transmitter after a pre-determined amount of time.