I understand why there is distortion if there is no opamp there, but not when there is an opamp with the output is connected back to inverting input of the opamp. The output would always equal to the input * gain no matter if the transistors off when input is -0.7 -- +0.7V? Is this correct?
Yes the opamp feedback circuit will try and correct the situation, but then the output of the amp isn't the real reflection of the input signal (+ gain) for short period of time around zero input signal. It causes a 'notching effect' at zero crossing that is easily seen on a scope and can be heard if one knows what to listen for. Again just read up on basic audio amplifier circuits using discrete components and you will learn all there is to know about the subject.
Modern IC audio amplifier chips do the same thing inside to eliminate the problem. Again this is a classic problem for any amplifier using bipolar voltage push-pull output stage, which allows DC coupling to the speaker. A single ended amp stage does not have this problem, but are very inefficient running in continuous class A operation (50% current flow with zero audio signal) and have to be capacitance coupled to the speaker.