LEDs losig brithness at a different rate, am I damaging them somehow?

Where did you source your LEDs from?
Cheap eBay stuff from the far east often comes out of the reject bin. It could be a badly doped batch you have.

Very nice work by the way.