How much can 328p handle?

See bottom of page 12.
You may have to flip the SPI bit order: