Is having capacitor going to make a big different on the stability of the circuit??
Yes a big effect.
I tested both (with or without capacitors) with battery they seem work fine.
How have you tested it, with a scope? Don't skimp on capacitors if you want a reliable circuit.