GRRRR... I can unplug the arduino entirely except for the MIDI cable and the audio output cable. And I still get the noise, but quieter. Something must be wrong with the breakout?
With the MIDI cable disconnected is there any noise?
If it is connected but you don't connect the wire to the Rx pin of the board is there noise?
If you remove the serial reading (and the begin call) of the sketch is there noise?
If the MIDI device is powered down is there noise?
The noise is lots of short clicks, like in the screenshot of the waveform, one click I would assume for each MIDI message sent. Turning the mod or pitchbend wheels sends a ton of messages, so the clicks turn into a buzz. But there are also clicks at note on's and note off's. So yes, no noise without a MIDI cable connected, but that's irrelevant, since the noise only happens when a MIDI message is sent.
I have the breakout board mounted as a shield, so I can't physically disconnect a wire. But there is a switch that disconnects Rx, so the MIDI circuit doesn't interfere with programming the board. And yes, the clicking/buzzing still happens with that switch disconnecting Rx.
Yep. Assuming the DAC is already in active mode, i.e. not floating, I can upload the bare bones sketch with the empty setup and loop, and I still get the clicking/buzzing noise. So absolutely an electrical issue.
Again, no, but not relevant, since the clicking only occurs when a message is sent.
I'm still puzzled... I tried measuring the resistance between the MIDI connector pins and various points on the board, but nothing unusual. It seems to be isolated.