Wow. Weird.
I was fixing my fathers old radio. It had a problem where only by touching the sound came out. I didn't understand what was going on until I dug out the datasheet of the ancient amp ic. It turned out the amp ic had 2 operational modes depending on whether 1 of the ic pins was grounded. When I touched the radio, the pin was grounded and the sound was output to the speaker circuit. The original contact to ground had been lost due to a dried out capacitor.
I have no clue what is wrong with your sound breakout, but I have had problems with cheap chinese modules where not all pins of the ic were properly tied to ground. The fact that your hand affects the sound output suggests to me that some pin or some ground that controls the playback is 'floating'. Just my 2c.