I wonder if you are having problems because you are exceeding the maximum allowable reference input voltage (4.18V). See the last line of page 3 of the datasheet.
--
The Gadget Shield: accelerometer, RGB LED, IR transmit/receive, speaker, microphone, light sensor, potentiometer, pushbuttons