Do you have a resistor at all?
A piezo doesn't provide a DC reference so without a resistor, the input will "float" (to an undefined voltage).
It should be a high-value resistor (Typically 1M Ohms or more) connected between the analog input and ground. The lower the resistor value, the lower the signal.
With the resistor, the ADC should read zero (or near zero) when the piezo is "inactive". With a high impedance/resistance input, you will pick-up some noise, especially if the wires are long and/or near any AC voltages, so it probably won't read exactly zero.