With nothing powered on there was continuity between vdd and speaker +.
I would not take too literally the + an - markings on the speakers. It all depends on the circuit of the amplifier output stage. When you measured the continuity did you do it in both directions? That is swap the leads around to make sure you were not going through a semiconductor PN junction.
However it is possible that the speaker is connected to the +ve rail of the supply and it is pulled down by the output stage. This will result in a square wave type signal.
For what I is worth I don't think your problems are with the 4.5V regulator either, mind you I think it will work just as well off 5V as 4.5V.
I had intended to connect the speaker + contacts to digital input pins and tie all of the speaker- to ground
So why not connect the speaker - to the arduino input pins and all the grounds together, would that not do it?