What's wrong with this simple PCB design? (midi synth)

I think the basic issue is that you are deriving an analog signal from a PWM pin which is on the digital
supply bus. You should try adding much more decoupling to the supplies on the 328, ranging upto 1000uF,
perhaps 0.1uF, 10uF, 1000uF.

There is no point separating the analog and digital supplies on the 328 chip itself, you are not using the
ADC or analog comparator.