Which Arduino are you using?
as soon as I saw smoke I completely unplugged everything.
Identifying where the smoke came from would be helpful in your troubleshooting.
So basically I plugged the 9V supply into the 5V pin
None of the active ICs on the Arduino board are tolerant of 9V.
Remove the ATmega328 as a first step. If your computer will recognize the board then you know it is bad but the USB to Serial converter is still working. If you have the components to do so, try building a Barebones or Breadboard Arduino with the ATmega328 to see if it works standalone.
Use a multimeter check to your 5V and 3.3V outputs when connected to the 9V supply. Keep in mind that if the regulators are damaged under no to light load, they may be stable. However they may no longer supply their full output capability anymore.