SPI_CLOCK_DIV and internal oscillator

Can you show your sketch? Sounds more like a programming issue.

The clock speed /accuracy should not affect SPI at all. Unlike serial comms, the clock is explicit - that is, you actually send the clock pulse.