Is it ok for a stepper motor to vibrate and make noise?

For drivers like the A4988 and DRV8825 the time that the step pulse needs to be high is on the order of 1us (microsecond). Read the data sheets. The time that the pulse is low is variable and will determine the speed of the stepper.
See also, Robin2's simple stepper program.

OP, steppers can be noisy. Have you tried microstepping? That can help. Microstepping will also mitigate the effects of resonance, which can be noisy as well as make the stepper miss steps.

There are drivers like the TMC2209 that will be quieter.