Has the library the ability to play wav-files ?
maybe you can try SPI flash solution,it will response faster than SD card.
... especially with no gaps at loops? ...
I have had problems with theese chips. (still have)I came over an idea (cant remember where) that 3,3V may be just to low to make this work.A cellphone battery (3.6V) as powersupply had make it work as a standalone unit.