It allows me to enter in the first number on the first switch, and then when I click the second switch it allowed me to input the second number with the second switch. Then when I hit the second switch again, nothing happens, and if I hit it again my program just restarts.
Is the bit in bold above correct? It doesn't seem to meet the requirements.
If it was me, I'd remove a lot of the duplication in that sketch by defining a function to input a number on switch 1, that ended when switch 2 was operated. This would be easy to test in isolation. THen your sketch would just call this twice, multiply the two numbers together and display the result.
I would also hold the numbers just as plain old integers instead of your array of trinary characters, and I'd do the number-to-trinary-display-format conversion at the point of displaying the number. You have to implement this anyway to display the answer, so you might as well do it to display the entered values. (This is easy enough to do by using div and mod to get the least significant digit repeatedly until the remainder is zero.)
IMO the crucial thing to keep in mind is that binary, hex, trinary, decimal etc are all just ways to display the number and do not change the number itself - you don't need to define a special representation of trinary numbers and write your own arithmetic operators and so on - they are just numbers, and only become trinary numbers at the point you format them (in trinary) for display. I would guess that your six LEDs are intended to provide three trinary digits each consisting of 0, 1 or 2 LEDs on. But maybe you have some other cunning way in mind to represent trinary digits using LEDs?