Feasibility: LED Matrix Shield, RTC, Buttons and Knobs

I'm interested in creating a color clock project. I'm comfortable with the programming, I can do light soldering, but I don't know anything about electronics. So I'm looking for a solution where I can put the hardware together easily and then just write the code.

My main question is:

  • It looks like if I use the RGB LED matrix shield to attach an LED matrix, it uses up all my IOs. I still need to attach a real-time clock breakout board, two buttons to set the clock, and a button or knob for brightness. How can I attach all this stuff together without things getting complicated?

I'm not a hardware wiz, so I'm hoping there's a simple way to make all this stuff work together. Anyone have any advice?