Nick, when you say they have internal pullups on them, couldn't that be disabled? Or does the arduino code keep re-enabling that (or are we talking about the 32u4 connected to them?) If serial is never invoked in your code, I don't understand why the pins would be pulled up. But, wouldn't simply setting them as inputs and writing zero to them disable the pullups?