4k7 pull-up resistors by themselves usually won't cause damage (the current is limited by the resistor).
There is a sweet spot for the resistance on the I2C pins, and from memory 4K7 is about right. But if there are already internal pull-up resistors built in and you add external pull-ups they will be in parallel, so total pull-up resistance is reduced and thus current draw on the pin can increase when active low. Maybe this was a little too much current for the Due .... (depends on total current draw on the Due for your whole circuit)