I wonder if the on notes and hence LEDs are falling between the cracks. That is if the duration of the note is less than the time it takes to reload the data into the matrix driver.
Just as a test. Comment out the LED off part and see if those notes are actually being caught.