Cylindrical versus toroidal coils in Class D amplifier

A "Class D" amplifier is an audio amplifier with switching mosfets. The switching frequency can be from 100kHz to 500kHz.
Sometimes the switching frequency is slightly filtered with a capacitor and passed on to the speaker, but in many cases a filter with a coil is used.
The PAM8610 module I bought on Ebay doesn't seem to have a coil at all, and has reasonable quality.
My Scythe Kama Bay Amp Mini has interference/noise problems, and it turns out that is has coils with ferrite cylinder cores.
Beside grounding problems and power supply interference (I can fix those), I have never seen cylinder cores with a class D Amp, only toriod cores. The cylinder ferrite cores can be used to filter RF signals, but this is a powerful signal.

My question: The output of this amplifier is 12V switching with 500kHz, would the cylinder ferrite cores be useless ? Would toroid ferrite cores be better ?

That all depends - without the part numbers and datasheets there's no way to compare
those inductors. For a LPF you need the right inductance value, enough current handling,
and placing the inductors so they don't mutually interfere (for solenoids that means placing
them at right angles or a long way apart.

Toroids have the big advantage of not having to be separated by much distance since they
have only small fringing fields. However they have much less stable inductance as they
have no gap(*) so the inductance is dependent fully on the ferrite (and thus varies with
temperature and DC bias). For a class D filter such unstable inductance value isn't an
issue but for a tuned circuit it would be useless.

(*) gapped inductors owe most of their properties to the geometry of the gap, not
the permeability of the core, making them much more stable. They can also store
more energy before saturation as the gap holds most of the energy and gaps don't
saturate.

Find out what modulator IC the board uses and see if you can find its datasheet. I've been going through some switching regulator datasheets recently, which have A LOT in common with Class D amplifiers, and there's usually a fair bit of advice in there regarding parts selection and PCB layout. You can really learn a lot by reading a few different ones -- like similar ICs from different manufacturers.

I did read the datasheet. It is the Yamaha YDA138(D3).
The datasheet has very little information on the coil.
They mention Q=0.7 or so for the coil.