dlloyd:
I've never used a breadboard without at least a 47-100µF cap and 0.1µF ceramic cap across the ground bus and power bus. I would suggest using just two side by side power rails near the middle, then run separate power and ground connections where needed from there.
Darn it ... Grumpy was quicker.
I have a big electrolytic capacitor (1000uF) across the power and gnd lines elsewhere in the circuit. As I am sure is clear, I'm pretty unsure of what I'm doing with laying out capacitors! The logic is all a bit murky to me. Keen to learn though!
LarryD:
Will you verify the 22pf capacitors have one end going to GND?
I would add some 10 and .1 uF capacitors at several places on all power rails.
Shortening wires to a minimum will help in reduction of noise interference.
See this for making small jumpers. (use solid wire #24AWG)
FYI, wire stripping trick - General Electronics - Arduino Forum
Hard to see decoupling on the controller and the wiring of the reset circuit.
Yes there is a black jumper going from the 22pf to GND. Can shorten wires, however I'm a bit worried that if it's so sensitive to this kind of thing, then maybe i should look for other solutions. What are your suggestions on value/location of capacitors?
Wawa:
I see why you didn't want to post this before 
Pin14 of the Atmega chip (D8) should not be sensitive, because it's an output (low impedance).
If the green wire to the latch pin is sensitive, pin 14 of the MCU could make bad contact with the breadboard.
Can't you move the MCU one or two holes to the left, and fit the 595 next to the MCU.
And only have the LEDs on that second breadboard.
Leo..
I have tried using different pins for the latch, doesn't make a difference. Will try moving the 595 right next to the MCU.
Grumpy_Mike:
Do not connect any capacitor to the /OE line.
No.
Where are your decoupling capacitors on your stand alone board?
And who prescribed them? Another crap tutorial?
Solderless breadboard is crap for anything none trivial, the connections are not reliable enough. That layout is a mess.
Can you suggest a better tutorial for capacitors on a standalone atmega328?
I connected it to the /OE because that is going to ground and I can't stretch the capacitor from Vcc to GND (opposite ends of IC)
The decoupling ceramic capacitors on the atmega328 are across pin 22 (Vcc) and pin 20 (GND), also across pin 7 (Vcc) and pin 8 (GND).
I get that it might be more reliable on a PCB, with shorter connections etc. but I want to get something as robust as possible before I commit to that.
Is the shift register my only option? (compared to the decoder where I think I need 5 pins to control 8 LEDs - is that right?)
Thanks everyone