Is the amplifier input connected to the "Audio-L" output?
This is probably the DC offset* kicking-in but I don't know anything about the WTV020SD. It may be one of the control signals "switching on" the output. Do you have a multimeter to check the DC output? It should be sitting at half the supply voltage (even when "silent").
It's also possible that there's something wrong with your audio file(s).
I was trying to make a RC low-pass filter between the wtv020sd and amplifier but it's still the same. How can I get rid of this?
If it's the DC offset that won't help. DC is "zero Hz" so it will pass-through a low-pass filter. But you can't hear constant DC. You only hear something when it kicks-in and kicks-out. But, there should be a series capacitor in-between the two to block the DC. If you bought a PAM board/module the capacitor is probably built-in.
- The audio signal needs to switch positive & negative. Since there is no negative power supply, the output should be biased/offset to half the supply so it can swing relatively negative. Then the bias/offset should be filtered-out with a capacitor. With a single power supply there is almost always a click/pop when power is turned on & off, but there shouldn't be a click when the audio starts.