Currently using a microblaze with drivers on the FPGA and the UNOs I2C library. I have 4.7K pullup resistors.
This is going to seem like a simple question, but do I need to pull up both sides of the level converter too the reference voltage being used on that side?
Since I2C is an open collector pin, you would need a pullup on both sides. That's because the converter is in between and you can't rely on it to pull the pins to the correct voltage. Actually the open collector was supposed to adapt the bus to interface different voltages. Hence you can tie it to whatever the system voltage is. In your case, if the ATMega328P can detect digital transitions at 2.8V, then I would actually get rid of the converter and tie the I2C bus pullups to 2.8V. Definitely the AVR can detect transitions at 3.3V.