Why would a capacitor connected to a pwm output "average" the voltage??

The very pin delivering the signal is the charge/discharge channel.

The voltage over the cap will not be constant, but have a ripple, roughly a sawtooth. How much depends on the size of the cap and the total series resistance (which includes the internal resistance of the pin, the capacitor, the wiring, and whatever resistor is in between the pin and the cap). Stray inductance may also play a role. Connect a scope to see this ripple. The cap charges during the high part of the PWM cycle, discharges during the low part of the cycle.

Your multimeter will indicate a constant voltage as it averages out the ripple.

As a follow up to wvmarie's post and the mention of the ripple please note the below image. The PWM is a 100 KHZ signal with a 50% duty cycle. You can see the charging curve of the capacitor against the signal. Also note the presence of the ripple component. The curve of DC level is not a nice clean line.

Bummer my image size was reduced. If you right click and open the image in a new tab it will look better and show the mentioned ripple.

Ron