H-bridge with peltier cell

Yes.
Without it, the thermal time constant is overwhelmingly slower than the PWM cycle, so the efficiency should be about the same for the same power.
e.g. simple resistive heaters.