Hi,
Its not the speaker thats at risk, its the Arduino. The 40ma limit is the most that a pin will safely output, it will output more if you let it, but it will die in the process. It will look like your getting away with it, but you are not, your damaging your chip and at some point the output pin will stop working.
I am not sure that the 318 op amp is a solution, its more of a voltage amplifier than an current amplifier.
If you don't have an LM386, you can try a simple resistor and capacitor network to send the output to an mp3 player or anything else with an Aux input - the circuit is shown in the Illutron B link above.
Duane B