INPUT_PULLUP question

I'm using an Adafruit Trinket (5V) with the following code:

// Pin 1 = LED 
// Pin 3 = output to remote control, active-high
// Pin 4 = input from switch, active-low, with pullup; use switch pins 1 & 3.

int led = 1;
int rc = 3;
int sw = 4;
char open_delay = 45;   // Number of seconds to wait until closing the door
long temp;

void setup() {
  pinMode(led, OUTPUT);     
  pinMode(rc, OUTPUT);     
  pinMode(sw, INPUT_PULLUP);     
}

void loop() {
  while (digitalRead(sw))  // Wait for sw to go low, meaning door is open
    ;
...
}

Execution blasts right through the while statement, so I checked the pin with a DMM and it reads 1.4V, which I take to be high-impedance. Regardless, I expected it to read 5V, so it doesn't look like the pullup is being engaged (it does go to ground when I press the pushbutton it's connected to). Here's a link to the pinout, and I'm probing digital read/write pin 4 (just below the Ground pin in the diagram), which is the one I'm setting up in the code. What am I doing wrong? Thanks for any tips.

take a look here, which gives you a pretty good understanding of LOW and HIGH voltage expectations based on you Vcc

That link generates an error.

Is it plugged into USB?

Pins 3 and 4 are used for USB, they won't work normally when it's plugged into USB.

Is it plugged into USB?

Jeez, I'm such a dufus. Thanks!