No, it won't. When the input voltage drops below 3.3V, the output voltage will also drop below 3.3V. Eventually the output voltage will drop to the point where the Arduino will no longer function correctly. Even after that, it will continue to draw some current from the battery. So the battery protection is needed