delay(500); // delay for deboucing
There isn't a snowballs chance in hell that your switch will bounce for that long.
Why do you need to read the joystick 4 times to figure out what to do?
Why are you using a joystick, when the speed is not proportional to the joystick position? Or, why isn't the speed proportional?
Why aren't you using the Stepper library? Or AccelStepper?
What, EXACTLY, should the switches do?