The noise is lots of short clicks, like in the screenshot of the waveform, one click I would assume for each MIDI message sent. Turning the mod or pitchbend wheels sends a ton of messages, so the clicks turn into a buzz. But there are also clicks at note on's and note off's. So yes, no noise without a MIDI cable connected, but that's irrelevant, since the noise only happens when a MIDI message is sent.
I have the breakout board mounted as a shield, so I can't physically disconnect a wire. But there is a switch that disconnects Rx, so the MIDI circuit doesn't interfere with programming the board. And yes, the clicking/buzzing still happens with that switch disconnecting Rx.
Yep. Assuming the DAC is already in active mode, i.e. not floating, I can upload the bare bones sketch with the empty setup and loop, and I still get the clicking/buzzing noise. So absolutely an electrical issue.
Again, no, but not relevant, since the clicking only occurs when a message is sent.
I'm still puzzled... I tried measuring the resistance between the MIDI connector pins and various points on the board, but nothing unusual. It seems to be isolated.