LEDs don't turn off [74HC595]

My shematics is EXACTLY the same as this (except for the capacitor):

However, when I upload any of the codes for two registers (including some of mine), after few seconds every LED connected to 2nd shift register turns on and stays like that. I'm trying to figure out what's wrong without any luck.

This means your actual circuit is not the same as the schematic. This could be due to loose connectors or miss wiring.
See if it helps when you put capacitors across the power inputs to the chip.

I've tried with and without them - same thing. Funny thing - The LEDs actually turn on when I go into serial monitor - not before that.

That will cause a reset of the arduino.
Does it work as expected if you don't turn on the monitor.

For some time yes and then they all turn on (except for the first chip which is working as expected). If I upload a code which sends zeros to registers and reset the Arduino - it works. Nothing is turned on. After I turn the serial monitor on the LEDs are on again.

Have you tried swapping the two chips over and see if the fault follows the chip?

Thank you. Replaced the chip and now it works.