Problem connecting DUE and MEGA 2560 with I2C

I was trying to connect a DUE and a MPEG 2560 with I2C. I connected pin 20, pin 21 and GND of the two boards together. I used Wire library. The problem is that MEGA 2560 is able to receive data from DUE, but when MEGA was sending data back to DUE, it always fails. Any one got any idea about this?

Have you got some kind of level converter from 3.3v to 5v and visa versa. The MEGA will see 3.3 as HIGH so that makes sense but the DUE will not like 5v. It will probably damage it.

This sounds a good explanation. Luckily my DUE is not damaged.