I initially upgraded for more pins and now I am using them all. For reference, it is a custom board not a mega board. The 1284P is in an entirely different class than the 2560.
I think the pull-ups are the culprit. The pins are usually all set to input and they need to be. If they are set to output every they interfere with the controller, which is sometimes the point but not always. I can provide the schematic if need be, as soon as I release the boards to the public it will have to be anyway, its open-source.