It should work fine. Most power amplifiers* expect a [u]line level[/u] signal (about 1V) and you can get that from the Due.
Test the amplifier first by plugging it into a CD/DVD player, or the RCA audio outputs from your TV, etc. A headphone output (from your phone or computer, etc.) will also work. (If you have a problem, troubleshooting/debugging is a lot easier if you know the amp & speaker are working.)
A headphone output is around line-level, but you might need to crank-up the volume. The main difference is that headphones are capable of driving lower impedance loads and headphone outputs always have a volume control... Some line-outputs have a volume control and some don't. Again... That's just for pre-testing the amplifier & speaker.
Power amplifiers are designed to drive speakers, as opposed to preamps which are designed to bring a microphone signal or phono pickup signal up to line-level.