Lithium batteries are charged using a Constant Current/Constant Voltage (CC/CV)method. That means they rely on the current delivered in the CV stage reducing until the charger decides the battery is completely full.
The only problem you may have is that because the Arduino is taking current out of the battery as it's being charged the charger may never actually realise that it's completed the charge and show "fully charged". That's unlikely to be much of a problem. It basically just means that for some of the time you may actually be running the Arduino from the charger rather than the battery.
Steve