How much can 328p handle?

For a serial load, the relevant information is Table IV on page 12.

The formula I gave is correct:

int32_t freq = frequency * 4294967296LL/125000000L;

This calculates the tuning word (given the misleading variable name "freq" here) from the desired frequency. The tuning word is then sent to the device.

Pete