I connected a piezo speaker to a ATMEGA32U4 (pin 9). The other piezo-pin goes to ground
When I power up the board, the piezo is completely quiet. I then play a tone using tone(9,500,1000);. After having played the tone (and also after having executed noTone(9)), there is a soft hissing noise coming from the speaker.
I suspect this. When the sketch first powers up, all pins are in INPUT mode. This high Z input impedance causes there to be no current flow through the speaker, causing absolutely no noise.
When tone() is run, the pin is forced to OUTPUT. After noTone(), the pin is still OUTPUT, but written LOW. This is a low impedance output that can cause some currents and noise to flow.
After noTone(), try setting the pin to INPUT and see what happens.