Yes. One amplifier per speaker speaker, or two stereo amplifiers for 4 speakers.
You can buy complete little amplifiers like this (or a bigger amplifier) or you can buy amplifier boards or amplifier chips to build your own.
At mid-frequencies (1kHz) and above you can get rather loud with 1 Watt. Bass frequencies take more power and bigger speakers/woofers.
But before that... Your schematic shows 10K resistors. Assuming 8-Ohm speakers, that makes an 8/10,008 Voltage Divider and I'm surprised if you can hear anything!
Try about 220 Ohms. That will give you about 20mA (Ohm's Law) which is about the recommended maximum current from an output pin. (The "Absolute Maximum" is 40mA.)
You're still loosing most of the voltage across the resistor but it's worth a quick try.
With no resistor (and no amplifier) you'll get excess current, you won't get the full 5V, and the Arduino will overheat, possibly damaging it.
Other than background noise, "quality" isn't much of an issue with square waves. ...If you over-drive an amplifier with a sine wave the you get clipping which is harmonic distortion. Worst case the sine wave turns into a square wave, and that turns-out to be 50% harmonic distortion.
With no resistor you'll get excess current, you won't get the full 5V, and the Arduino will overheat, possibly damaging it.
Not a problem with low power (like through a resistor) and the the cone is actually "pulled back" to the center when the output goes low.
Most amplifiers have a capacitor on the input to block DC (and convert it to AC) but to be sure you can add a capacitor in series between the Arduino and amplifier. 0.1uF is usually OK but you may need a higher value for low frequencies, and this also depends on the amplifier's input impedance. (It makes a high-pass filter and since DC is "zero Hz" the "DC component" it gets blocked while allowing the signal through.)
