Mattias,
You gave a good idea of what you want the behavior of your system to be. Now, let's turn that into all of the different "states" for your system.
-
CLEAR (door closed, alarm quiet, green light on)
-
BREACH (door open, alarm loud, red light on)
-
TRIPPED (door closed, alarm quiet, yellow light on)
For each one of those states, you have to decide what the next possible state is, and how to know it's time to go to that state.
-
CLEAR: if the door is open, go to BREACH
-
BREACH: if the door is closed, and ten seconds has gone by in this state, go to TRIPPED
-
TRIPPED: if the user clears the alarm, go to CLEAR; if the door is open, go to BREACH
You can make a separate function for each state, and then the loop() code should just call the right function for the current state.
Once you get this working, and you learn more about the keypads, there's a new state to add: UNLOCKED, where it's just like CLEAR for a few seconds but it won't ring the alarm if the door is opened AND closed within a few seconds.