The frequency slows down continously but takes time to spped up!

Doesn't compile for me:

sketch_apr08b.ino: In function 'void TC0_Handler()':
sketch_apr08b:9: error: 'REG_TC0_SR0' was not declared in this scope
sketch_apr08b.ino: In function 'void loop()':
sketch_apr08b:42: error: 'REG_PIOB_PDR' was not declared in this scope
sketch_apr08b:43: error: 'REG_PIOB_ABSR' was not declared in this scope
sketch_apr08b:44: error: 'REG_TC0_CMR0' was not declared in this scope
sketch_apr08b:45: error: 'REG_TC0_IER0' was not declared in this scope
sketch_apr08b:47: error: 'REG_TC0_RC0' was not declared in this scope
sketch_apr08b:48: error: 'TC0_IRQn' was not declared in this scope
sketch_apr08b:48: error: 'NVIC_EnableIRQ' was not declared in this scope

It would help to specify what board you have selected, this isn't supposed to be a guessing game.

And please don't use the variable name "l".

Can you see the difference between:

foo = 1;  // the number one
foo = l;  // lower-case L

I can't, easily.