Is there a non invasive way to do so?
I'm not sure what you mean. You'll need to wire-up to the speaker terminals. I suppose that's not quite as easy as connecting to a line-level signal with a Y-splitter bit it's still pretty-easy...
Measuring current is not a good way to do it because speaker impedance varies over the frequency range. Audio specs/measurements assume "constant voltage" (voltage independent of resistance/impedance) and the "nominal" impedance is used for power calculations. For example, when measuring frequency response you apply the same voltage across the frequency range rather than maintaining true-power. And, that's how amplifiers work. Plus, it's just easier to measure voltage.
Or is using a voltage divider the only way? In that case it would have to have a very huge resistance in order not to interfere with normal speaker operation, right?
Yes, Voltage dividers are typically in the 10K range anyway.
100W is 80V peak-to-peak so you'll need a 16:1 voltage divider for the 0-5V Arduino input (if you want to go up to 100W).
I'd also recommend adding [u]protection diodes[/u]. (The protection circuit requires a current-limiting resistor, but you already have a resistor in your voltage divider(s).
I've attached a schematic for the standard bias circuit. The two 10K resistors "appear" in parallel to to the audio signal so they present a 5K load, which you can use for the "bottom half" of your voltage divider. (Add another resistor in series with the capacitor.)
I wouldn't actually worry about "speed". With normal program material most of the energy is in the mid and low frequency range. Aliasing is NOT a factor. Aliasing & Nyquist are related to frequency and the amplitude measurements will not be affected.
If you want to measure peaks, or if peak measurements are acceptable, and if you don't mind adding additional circuitry, you can use a [u]peak detector[/u] (AKA envelope follower). That will "catch" all of the peaks (including any high-frequency peaks depending on the op-amp, etc.) and your software is easier because you can sample at around 10 times per second instead of thousands of times per second. (Of course, the over-voltage protection goes in front of the peak detector.)
One additional thought - Most of the time "at home", you'll be running at a few watts. So, you may want to have multiple ranges, or make it auto-ranging. (If you use a peak detector the input won't be biased so you can switch between the default and the 1.1V ADC reference as an easy way to increase/change sensitivity.)
...I've made a "Giant VU Meter Effect" and it automatically adjusts based on a moving-average so I get lots of "meter action" no matter what the volume. But of course it's totally useless as a meter. 
