Controlling speed of dc motor

Good documentation! What is the DC resistance of the motor? You probably want to use some flyback diodes. Diodes in the SN754410 are just for ESD protection they probably cannot handle the induction kick from the motor. Also, it looks like the leads of the capacitor that is connected to Vcc1 on the SN754410 are shorted together which isn't a problem but the capacitor isn't doing anything when it’s used that way.

Where exactly was the smoke coming from?

What are the values of the resistors? It's difficult to tell from the image. Any value between 1K and 100K should be fine.