A 12V stepper motor driven by an A4988 that is powered by 12 V is not a good idea.
The A4988 needs a much higher supply voltage than the rated voltage of the motor to operate.
Have you set the current for your stepper motor on the A4988 correctly? And did you place a bulk capacitor at the powerpins of the A4988?
A 12V stepper isn't a good choice for an A4988.
The A4988 stepper drive voltage is specified as 8 - 35V, so 12V is fine - except that this will severely limit the maximum step rate. This is because the inductance of the windings slows the build up of current until it reaches the set limit - it also slows the decay of current when the voltage is reversed.
If the step rate is higher than this maximum and the inertial load is small, the movement of the motor is unpredictable because the coil currents are usually quite different to those you expected from the state of the step and direction inputs.
As the previous poster says, set the current limits to those defined in the stepper specification and use a low ESR high value capacitor across the 12V supply. If the required step rate cannot be achieved when the motor is loaded, then you have two choices - use a motor with a lower voltage rating (3.3V and 5V are common), or increase the supply voltage. Worst case you may have to do both!!.
To achieve maximum step rate under load, controlled acceleration and deceleration is necessary!! This is a whole new ball game and not simple to implement!!
Thanks for the reference to Accelstepper - being new to Arduino I wasn't aware of that.
My only experience of accel/decel is on PIC24F running at max speed - actually used a sqrt calculation to drive the maths. It was a hobby project and this was a learning exercise for me in programming the 24 series PICs.
Stupid me How I could not see that I connected motor wrong way. Adding capacitor and correct motor connection did the trick. Many thanks to you all.
BTW, anything I can do to reduce very high motor tempreture?
That probably means that you do not have the coil current set properly on the A4988 driver.
To set the coil current properly you must know the value of the sense resistors on your modules so that you can plug their values into the equation to compute the proper setting for Vref. See the Pololu A4988 product page.
That said, steppers can run hot. Hot enough to be uncomfortabe to hold a finger on. Burning the finger could be too hot.