I think I'll leave it alone for now and let it use 400KHz. At least now I know what's going on.
It's my understanding that with higher bus rates, the PCB traces are more critical. I have a PCB board getting made now where the I2C signals run across it to a few devices. I did a little research on how to lay-out high speed signal traces, so hopefully I did a good job. I'll see how well the PCB performs when I get it back in a couple weeks.