Does this mean the chip already on the arduino can pop off?
It depends on which board you get. The official duemilanove, diecimila, and many clones use a DIP-style ATmega168 or ATmega328p that is socketed can can be easily removed and put on a different circuit board. Some of the newer and/or smaller systems (nano, "arduino pro" from sparkfun, "Seeeduino", Arduino Mega) use a surface mount chip that cannot be removed.
The jumper they're talking about is the USB/external power jumper on the diecimila version of the Arduino that was eliminated in the current duemilanove board.