Does bandgap measuring cost extra power?
The ADC needs power, yes. Luckily for us Atmel thought of that and you can turn parts of the chip on/off on demand.
The AVR standard library has functions for this:http://www.nongnu.org/avr-libc/user-manual/group__avr__power.html
If you're using batteries you should turn off the parts of the chip you're not using. The ADC converter and Timer1 are the worst power gobblers (see datasheet).
It looks like my tiny is working such low voltage.
Don't ever depend on "it seems to work". It might be right on the edge of failure and start giving random errors a week from now when the battery voltage drops by 0.01V.
It looks like my tiny is working such low voltage. Should that V be printed on the chip itself (or a serial)?
It should say "ATtiny85V" on the chip.