Use a P-channel MOSFET. Drain to the OUT+ of the charging module, source to the load. Note that this orientation will make the body diode forward biased when current is being drawn from the battery. This is intentional and important. Connect the MOSFET gate to GND with a pulldown resistor. Connect the positive output of your wall wart connector to the input of the charging module, to the MOSFET's gate, and then through a diode (usually Shottky) to the load.
When the wall wart is disconnected, the gate's pulldown resistor brings the gate down to 0V, turning on the MOSFET. The diode prevents the battery voltage from backfeeding onto the gate resistance, which lets the MOSFET stay on. Power will be drawn from the battery with minimal loss.
When the wall wart is connected, the MOSFET's gate gets pulled above the battery voltage, turning it off. Power is supplied to the load through the diode, and the MOSFET prevents that from backfeeding directly into the battery. The charging module is able to charge the battery with the properly controller application of power as required, and the wall wart will supply both the charging current and load current.
This is a slight simplification of the circuit shown in Figure 2 of Microchip's application note
AN1149 - Designing A Li-Ion Battery Charger and Load Sharing System With Microchip's Stand-Alone Li-Ion Battery Charge Management Controller. The noteable change is the remove of the dual common-cathode diodes. Those are only necessary because the circuit is designed to take power from 2 external sources, USB and a wall plug, so they each have to be protected from each other.
A diode could be used in place of the P-channel MOSFET, but a turned-on MOSFET has much lower voltage loss than a diode and will waste less power.