SOLVED! ---
In order to stabilize the circuit I needed a Single external pull up resistor (not sure why it only took one) on Pin 3, which is Row 1 for both the 1x7 and 4x4 matrixes. Is given how I read the keypad code I should not have an issue with instate changes as if I am reading correctly the columns are the pins that will switch states..
Anyway It works, I'm Happy, if I am missing something here that can cause damage to my circuit or a thought of a better manner in which to do it please let me know.