More or less, yes, luck. Or, the Due is pulling the level down (this draws current which is what leads to catastrophic failure). Sometimes if the source impedance of the other end is really high you might be OK because the 3v3 side will then pull the level down and everything will work so long as the other side is OK with the signals coming back to it being 3.3v as well. But, in general you should not do this at all. If the voltage levels don't match then use level shifters. If you need slow speeds you can use resistor networks and transistors to do level shifting. If speed is important then use logic gates. A 2 input AND gate is a good choice. Both inputs go to your signal then you power the gate with the proper voltage for the receiving side. This conditions the signal with very low skew.