Not sure what you meant by use fricking code tags.
Anyway, I had double-checked the wiring several times and decided to check again. Turns out, I was actually missing GND to the chip.
What is interesting is the chip was actually doing things without ground.
I think one of my reasons for asking about the chip itself and if there is some sort of default, is it does have an EEPROM and I just wasn't sure if there are different chips out there that need some sort of initial programming first.
In line, for example, ATMEGA328 chips can have all kinds of different boot loaders or none at all,