I moved the 3v3 and GND wires to the Due for the TFT and TMP117, connected all the SDA/SCL wires and connected them to the Due without any PULLUP resistors and it still wouldn't start the TMP117. The TMP117 loads first in the code so it stops everything from booting up after power up.
I also tried it with the internal PULLUPS on pins 20,21 with no change in the results.