Dimming the SAA1064 below 3mA

So the SAA1064 is a very popular chip to drive 7-segment LED displays. You can also set the brightness of the display by setting the LED current between 3 to 21 mA. What I am running into is I need to dim the led below what the 3mA output would give me. The circuit that I am running is

What I was thinking to do is PWM the 5v supply going into the NPN transistors. And then just make sure the pwm frequency is an order of magnitude higher than the multiplexing frequency.

I know that this does allow me to dim the displays but would this damage the LEDs or the SAA1064?

So I made a Circuit like this:

Resistors R13 and R14 are to prevent the multiplexing signal from driving the LEDs
Resistors R17, R27, and R28 are to pull the voltage low when the PWM is off.