Indeed, why would SDA1 and SCL1 start to climb as I increase the value of their pullups? They should stay at VREF1 = 3.3V, but they don't. Note as well in this situation that VREF2, measured at the EN pin remains at very nearly 4V (where it should be, given that the supply is 5V) even as the pullups are changed, so it doesn't seem that fiddling with the 200k resistor is the answer either (although I admit to exhaustion rather than exhausting all possibilities).
Perhaps 3.4 should not be a worry, but in any case, the communications do not work when the 270pF caps are removed.
My main point here is that the PCA9306 does not maintain VREF1 if the pullup values are increased, so it may not be possible to tune the pullups on the Due side to the desired value. On top of this, the PCA9306 also makes the Due side bus capacitance visible to the other side, so adding caps to the Due side is not a good option either. So you are pretty limited in what you can do on the Due side if things aren't working.
Doug