You're right. Here's the relevant quote from page 50 of the ATmega8 datasheet:
"If PORTxn is written logic one when the pin is configured as an input pin, the pull-up
resistor is activated. To switch the pull-up resistor off, PORTxn has to be written logic
zero or the pin has to be configured as an output pin. The port pins are tri-stated when a
reset condition becomes active, even if no clocks are running. "