You have it backwards. The fuse settings (clock setting) affect the speed at which your programmer (an Arduino running the ArduinoISP sketch) can communicate with the target (your ATtiny85 processor). In your case, if you fail to lower the communications speed the target will not respond; it will appear to be bricked.
...even if the IDE said the burn was successful.
Fuse changes take effect after the target has been reset. From the factory you will be able to change the target's clock setting to 32.768 kHz clock crystal using the ArduinoISP sketch. If that step goes well, after the target has been reset (after avrdude has finished changing the fuses) the target will be running at 32.768 kHz. Until you modify the ArduinoISP sketch you will no longer be able to communicate with the target.