Thanks Lefty. It's actually better if you wouldn't mind nit picking the wiring diagram (if it's not the ugly soldering...which I can't help much) for the potential problem.
I also made one with capacitors flank at both upstream of 7805 input and downstream of 7805 output. Is having capacitor going to make a big different on the stability of the circuit??
I tested both (with or without capacitors) with battery they seem work fine. But not very sure when they are hooked up to the real device that sends 28V. What you will do to use the power source I have (0V and -28V) in this case??