Staying in a subroutine

Well, you only call menuselect() once, then you enter an endless loop. So no surprise there...

Here is how you might stay in a loop during setup:

bool runningSetup = false;
void loop() {
  //menu select programming:
  if (digitalRead(setupPin) == HIGH and runningSetup == false){
  runningSetup = true;

  if (runningSetup == true){
  menuSelect();
  }
  else {
    //run mode program
  }
}

If you want to remain frozen in menuSelect, you can perform that action there, or you have the option of running your program by setting

runningSetup = false;

again

Programmatically, it seems like you have designed things "bottom up" and then had an assortment of parts that don't fit together into what you want. By designing "top down" you can avoid that.

Actually, this solution was already given in reply #2.