Using Leonardo to read switches and write LEDs over USB

I am building a box which will have a bunch of toggle switches, pushbutton switches, and LEDs which can be controlled by a host computer over the USB port. The host computer program will be Python or C. I bought a Leonardo for this. I've tried the keyboard library, but it's not ideal since it puts junk in the keyboard buffer, doesn't place nicely with non-momentary switches, and doesn't expect to receive data for LED display. How to do this? Do I need to write my own low-level driver??? Do I use /sys/class/ or /dev/ ? Do I need a different board (Due, Micro, Teensy, or CY7C68013a?)

Thanks.

I bought a Leonardo for this. I've tried the keyboard library

Which sends keystrokes TO the USB port/PC. Why is that relevant for receiving data FROM the PC?

Do I use /sys/class/ or /dev/ ?

No.