Try connecting a resistor of about 33K between the output and ground. That should make the device perform better output go back to zero quickly.
The reason for the high pass filter is that the output will have a bias voltage of around +1V on it. You could connect the output direct to the Arduino (without the high pass filter, but with the 33K resistor in place), but you would have to allow for the bias voltage when you take the reading, and the bias voltage may vary a little with temperature. However, you would be able to read negative acceleration as well as positive acceleration.
Another way to read both positive and negative acceleration is to modify your high pass filter to add a bias of +2.5V. To do this, replace the single resistor you have between the input pin and ground by two equal resistors, one to ground and one to +5V.