Im new to all this..so take my advice for what its worth!
I dont think 'controlling' a large amount of LEDS is the bottleneck here.. as that can be overcome by shift registers..etc..
I think the 'problem' will come into play because you want a button to go with each led..
so 100 leds = 100 buttons...etc..
by default.. you wont have enough I/O pins on any current Arduino...
that being said I believe I have seen solutions or posts on using/having a MATRIX of buttons.. (which Im guessing is similar on how large matrix of leds are controlled?)
Id say the first search topic is finding a solution to MASS INPUTS that your ok with..