First off, I will say that building an amplified speaker using an Arduino would be like attempting to bake a cake using the ingredients for a pot roast dinner; it might be able to be done, but the results will likely be less than desirable...
rogerClark:
You can't do this without external hardware, on a normal ATMega based Arduino, e.g, a Uno or even a Mega
Well - you could sample 1-bit audio, and if played back properly, you could even get a recognizeable signal back out of it; I used to play with this quite often on an IBM PC, where you could redirect one of the timer interrupts to trigger an ISR to sample audio from the cassette input port (back when you could load tapes on the PC - this was decades ago, of course). You couldn't get many seconds (maybe 3-4 tops) for the memory you had, and the output quality was not that great, but it did work.
I wouldn't doubt that you could do something similar with an Arduino - and with maybe a simple SD card hooked up, you could sample for quite a bit (even at a higher bit density). But again, that would be extra hardware, so you have a point there, which you note:
Firstly they don't have enough memory to record audio.
Normally people use an external SD card to store audio for this sort of application
rogerClark:
Secondly the normal Arduino's Uno, Mega etc, don't have any hardware to convert the recorded audio back to an audio signal again eg for playback.
Well, you would need an ADC to get the audio into the Arduino (again, a one-bit audio signal could be done with a simple comparator) - but those aren't easy to construct from discrete components. To get the audio out (again, for something more than 1-bit) you would need a bit of hardware, as you mention, as a DAC (digital-to-analog converter) - which can be fairly easy to construct from simple parts.
rogerClark:
You'd need to use an Analogue to Digital converter IC to do this, or at the very least build a simple resistor chain to do the same thing (however a resistor based A2D would not give very good quality and would be hard to write effective software for.
Actually, the quality can be quite good, provided that all the parts are carefully selected and matched (1% tolerance resistors, carefully matched to near-equal tolerance levels, for instance). 8-bit audio isn't that bad, even sampled - it's what the Amiga output, and more than a few artists made quite great MOD tracker music with some nice samples (including voice). Take it up to 10-12 bits (still doable with a handmade resister ladder DAC), and coming from a mono speaker, while it won't be audiophile quality, it won't be terrible by a long shot, either.
rogerClark:
Thirdly, the regular Arduino's (Uno and Mega etc), don't have fast enough CPU's to perform Auto Tune e.g Vocoder functions.
You may be able to speed up or slow down the audio playback but not pitch shift without affecting duration
While the Arduino wouldn't have enough memory to store a large buffer of sound, nor would it have enough speed to do more complex effects - I do think with proper coding (which would likely include some hand-tuned assembler, direct port writing, and not using the Arduino libraries) you could perform some very interesting audio stream processing (perhaps with a bit - or more than a bit - of lossy artifacting of the output). It might even be that the output, with whatever distortion or other "funkiness" that it exhibits - might actually have a nice quality all it's own, much the same a "circuit bent" audio devices.
rogerClark:
The Arduino Due has A2D built in and is much faster (not sure about the Yun), but neither of these boards as for beginners, and they are fairly new and have a lot less software written for them and less community support at the moment.
You may have better luck if you used a Raspberry PI, as that has bags of ram audio in and out and also a fast CPU. But its an entirely different sort of device to the Arduino's
No argument here for these suggestions - but again, I do think that the Arduino could be used as a simplified audio stream processor - with a certain amount of minor extra hardware on the input and output side - provided that proper programming and certain concessions were expected.
Regardless - it wouldn't be a beginner's project at all...