The battery has internal resistance. This value is relatively low when the battery is fully charged and rises as the charge drops. Your load resistance is in series with this internal resistance and hence the battery voltage at the output terminals is dependant upon , not only the state of charge but also the load applied. Since there is no current flowing with the load disconnected, no voltage is "lost" across this internal resistance and so the voltage measured is sonmewhat higher than that with a load.