Yes very true.

I always say that Physics is the sub branch of mathematics that happens to deal with reality.

Its like the difference between believing we invented complex numbers as opposed to discovering them.

No we actually invented them, they do not exist, they were not discovered. It turns out that they are amazingly powerful at describing a lot of things but they are a construct of the human mind. You can imagine some alien civilization inventing a way of describing the universe in terms of smell in a rigorous logically consistent way that would be similar to mathematics but would have no numbers or even the concept of numbers.