Okay. Thanks everyone for all the help.
I downloaded the 1.5.6-r2 version of the Arduino IDE and it now works like I would expect. Based on a conversation in another thread Due analog read · Issue #1740 · arduino/Arduino · GitHub it looks like this was a bug where adc_disable_channel(ADC, ulChannel); was not properly called when channels were switched.
Man, that was a lot of time down the drain for a library bug. Ugh.
But working now! Thanks again for helping me pin it down!