Well, if we're going in that direction, you should also consider the new Robodyne and WeAct STM32F401 boards in the Blue Pill form factor that are out now. There are also new boards with the STM32H7 series, they are extremely powerful. The 401 is ridiculously cheap for what it does, the H7 boards are pricier but still not too bad. ESP32 is an option too, but I think it won't be conservative with power, by default. You would have to engage some software controlled power management. Pick a processor that has an FPU.
The Due memory is in two banks, 32 and 64k. Your arrays should still fit in there, I suggest investigating further before giving up on the Due. Again, you didn't say whether it compiled? ran? crashed? produced incorrect results?