SPI protocol is defined as "4 wires" bidirectional communication.
You would be replacing physical wires with infrared channels.
Do you want bidirectional communication?
If not , you could use just three wires AKA three infrared channels or come up with converting them into serial communication on single channel.
In short - doable with some effort.
Cheers,