I would just try it myself but it's turning out to be harder than I thought to find something to create
the int array so I figured I'd ask if anyone thinks this would work before I go to all the trouble of likely
writing a little program to do the conversion for me.
Yes, it's possible, but you're going to run out of memory very quickly. Audio files, even 8-bit/8KHz (poor telephone quality) take 8KBytes per second. You might consider creating your audio files with Audacity, which is available at http://audacity.sourceforge.net/
There is a technique that I saw a while back for PIC microprocessor, but could be used with Arduino, to encode the audio as a 1-bit stream that used a much smaller memory area. You can see the article at BTc Sound Encoder v3.0 - free sound software