Can you damage the ADC with a voltage above Vref?

IIRC you should not apply voltages above Vcc, voltages between AREF and Vcc are mapped upon MAXVAL/1023

To be sure check the Atmel datasheet