When connected to the USB port the onboard 5V regulator doesn't do anything as the computer is supplying the 5V.
When using an external supply through the power jack the 5V regulator is then working. As it is just a linear device (cheap and supplies a nice 5V ) it is also somewhat inefficient. As your supply voltage goes higher the device wastes some of the energy passing through it as heat. The higher the voltage, the higher the heat. The more current you are drawing from the regulator, the higher the heat. This is normal operation for this type of regulator.
How much current are the attached devices drawing?
To reduce the amount of heat generated you can power the devices attached with a different supply ( and it will also generate some heat...), just tie the grounds together.