You say "I am powering one 0.4A 12V stepper with it but the regulator is getting hot after a while and switches off". Not clear what you mean here? Are you talking about the motor controller board? Or are you talking about the Arduino?The schematic diagram for the motor controller is ambiguous about how its power relates to Arduino, but it doesn't look like any motor power is being drawn through Arduino. Do you have anything else connected to Arduino? I would not expect Arduino to overheat and switch off unless you had additional loads connected to Arduino. It should tolerate being operated from 12V into the VIN pin or DC input jack.
If you want to use the DC jack set your regulator to 7.5V (there is a diode in line with the DC jack)If you want to use the Vin set your regulator to 7V (no diode)If you want to use the +5V pin set your regulator to 5V. (no regulator)All are good ideas, the first offers the most safety, the last the best efficiency.
Here are the tests I would do if I were getting your symptoms....[ ] Yes [X] No Does the Arduino heat up and shut down with the motor controller shield completely disconnected?[X] Yes [ ] No Does the Arduino heat up and shut down with the motor controller shield connected but the motor disconnected?[X] Yes [ ] No Does the Arduino heat up and shut down with the battery connected directly to the motor shield?