Problem connecting DUE and MEGA 2560 with I2C

Have you got some kind of level converter from 3.3v to 5v and visa versa. The MEGA will see 3.3 as HIGH so that makes sense but the DUE will not like 5v. It will probably damage it.