I've also tried it with condition "while(customKey == "B")" to exit the loop but it does not work.
the key "B" will ask me to re-enter the password then choose to arm (turn gas sensor and sound sensor on)system or disarm(turn both sensors off). but it just stays inside the loop until i take power off the arduino or press the reset button on the board.
If you want a responsive program then don't use WHILE. Just allow loop() to do the repetition and then you can check the I/O pins on every iteration of loop(). Have a look at how it is done in Several Things at a Time
The value of customKey never changes while your do/while loop is running, so expecting the do/while to ever end is unrealistic. You need to call getKey() now and then, in the body of the do/while loop. Or, as advised, get rid of it.