You can drive a TLC5940 to death no problem, I have done this. I was driving an 8x8 LED matrix - all LEDs at once and cycling the duty cycle from 0 to 4095 and it would go flakey on me after a minute or so like you describe and I killed a couple of chips too. When I made it cycle from 0 to 2048 it resolved these problems.
If your numbers are right and you are sinking 320mA @ 12V. 320mA * 12V = 3840mA but DIP package is rated for 2456mW total.
I am not sure why it would be a problem for the second IC and not the first if they are dissipating the same amount of power.