Whatever I will suggest will boil down to implementing what the keypad library does: a function to monitor key states and presses and return the value.
So if you don't want to use the library, extract the code from the library...
Once yo have a getKey() function that returns a key, you could have a look at the code in this post (thread is in French but code is code) and you'll see how I collect keys into a buffer for further processing once you hit the end marker '#'