The Arduino has pullups, which ties your inputs to VCC with ~ 20K resistor.
Enabling them will ensure a constant value, but this really shouldn't be a concern. All pins in input mode are high impedance. See THIS article:
Pins Configured as Inputs
Arduino (Atmega) pins configured as INPUT with pinMode() are said to be in a high-impedance state. One way of explaining this is that pins configured as INPUT make extremely small demands on the circuit that they are sampling, say equivalent to a series resistor of 100 Megohms in front of the pin. This makes them useful for reading a sensor, but not powering an LED.
As for the stabilizing cap.. There is no need when run 100% battery power. Your batteries are not fluctuating wildly like an AC adapter. The caps that are built in to the Arduino on the 5V rail are more than adequate. You could desolder all of the electrolytics and not notice when running on a battery pack.
Now, try that with a wallwart and you'll most certainly fry things. They have spikes leftover and must be dealt with in the form of the electrolytic "stabilizing" caps.