GPIO r/w on Due (84MHz) slower than on a M0 (48MHz)

So I must therefore also resign with this issue

You can always write a faster version...
See also https://forums.adafruit.com/viewtopic.php?f=57&t=133497