I checked both crystal using my oscilloscope, everything appears to be normal. I used the same crystal with the same resistor as the Arduino board (CSTCE16M0V53-R0 16MHZ (built-in capacitor) with 1M ohm resistor). The blink program makes the LED blink at the exact timing (1000 ms on, 1000 ms off).
I tried to program the 16U2 using the hex file on my Uno and its fuses settings, no improvement. Always exactly 9,5 seconds (within the precision of me using a chronograph !).
I may have missed something... If someone has an idea, I'm listening 