Recently I am planning to use attiny 85 powered by a CR2032 coin cell battery to light up some LEDs randomly. I tried to connect them but it seems that the LEDs is actually not that bright.
A CR2032 has very limited current capability, maybe only 4mA.
a) You won't need a resistor with the LED when you're on battery power.
b) You'll need to reduce the current used by the Tiny85, this makes more available for the LED.
Disable all the parts of the chip that you aren't using, especially the analogue parts. Functions for doing this are here: http://www.nongnu.org/avr-libc/user-manual/group__avr__power.html
See the section on "power management" in the datasheet for more info (also the "Supply Current of I/O modules" section, under "22. Typical Characteristics").