I charged the battery before the test and connect the battery to the JST port of the Arduino board.
When the board is connected to the USB, both the ON LED and the CHRG LED turned on, which I assumed that the battery was fully charged. However, the system didn't work after I disconnect the USB (the ON LED turned off). It seems that the board cannot be powered by the battery.
Does anyone know the potential reasons or face a similar problem before?
The lithium battery you reference has a built in protection circuit, I would test the voltage at your JST connector on your battery first, if it reads 3.7V or higher(though 4.2 is max) the battery/protection circuit is most likely fine(could be as low as 3.3 and the arduino will still work, because it appears everything is powered from the 3.3V regulator, anything less will most likely cause the regulator to cut out). Then, with the battery connected to the arduino test again(without USB connected) if the voltage is still above 3.3V, but still no ON led, then a malfunction is occurring within the battery charging circuit, maybe the FET Q3A that isolates the battery from the USB 5V when connected. To test this, you could test the pins at the regulator U2, pin 2 with the battery connected should have battery voltage(whatever you battery reads approx.) in relation to the GND pin 1, if no voltage is present, then Q3A is malfunctioning. If you have battery voltage there, test the regulated output on pin 3, should be almost exactly 3.3Vdc. Download the schematic for reference. https://store.arduino.cc/usa/arduino-mkrzero