You need at least a 100uf cap on input side of the LM7805 in this application as well as a 0.1uF on the output. Also, as already suggested put a 0.1uF across Vcc and Gnd of the ATtiny. If you can get a hold of a 50mH to 100mH choke rated at twice the current your circuit draws would also be very nice and quite possibly a requirement. I am assuming your circuit draws less than about 100-200ma. Those 4-amp automotive radio filters are pretty bulky.BTW, what kind of bike is it?
I have blown up 78L05 regs before by not using input capacitors