Maybe the library documentation can be improved slightly based on the answers to these questions:
(1) As background, my device's datasheet (page 18) says that its input shift register is 24 bits, with the first 6 bits ignored, the next 2 control bits controlling the "mode of operation," and the last 16 as data bits in order of Most Significant Bit (MSB) first. I don't know the extent to which these are SPI implementation details versus part of the device's "public" API.
With that, is the SPI library's transfer() function just for the data bits, or for the full 24 bits? If the latter, what is SPI.setBitOrder() for since the caller needs to pass the bits in manually anyway?
(2) Is it just a coincidence that the device has four "modes of operation" specified by two control bits (page 19 of the datasheet) while the SPI library's setDataMode() also has four modes specified by two control bits? The descriptions of the modes seem completely different.
I don't have access to the hardware yet to try things out, so I'm asking in advance.
With that, is the SPI library's transfer() function just for the data bits, or for the full 24 bits?
You need to transfer all 24 bits.
With that, is the SPI library's transfer() function just for the data bits, or for the full 24 bits? If the latter, what is SPI.setBitOrder() for since the caller needs to pass the bits in manually anyway?
The bits might make sense to you in one order, and to the device in the reverse order. The SPI library can manage the conversion, if needed.
Is it just a coincidence that the device has four "modes of operation" specified by two control bits (page 19 of the datasheet) while the SPI library's setDataMode() also has four modes specified by two control bits? The descriptions of the modes seem completely different.
SPI.transfer works only with a single byte at a time. setBitOrder will transmit the bits of that byte in the requested order, if the value you are sending is more than one byte long it is up to you to transmit the bytes in the correct order.
The setDataMode modes are for clock polarity and phase. While I haven't looked at your datasheet I don't see any way that they can be related, since you must already be communicating with the correct polarity and phase in order to transmit data to your device in the first place. They must be modes specific to the operation of the DAC.