Arduino Yun: SD not working with battery

I am using Arduino Yun to store data form an IMU into an SD card.
I am able to store data by powering Arduino with USB
But when I connect it to a battery, the file to store data is not getting created in the SD Card.
The problem happens when the code is uploaded through wifi and also when it is uploaded through USB
I don't think the problem is with the battery since the led blinking programs work on battery connection.