Your latest purchase

Making an SD card adapter is pne kind of project you described.

My immediate goals are along the lines of blink an LED, send "Hello world" to a terminal program, connect something using I2C, something using SPI, etc.

According to ATMEL AVR docs I've seen, the hardware serial ports are capable of master-mode-SPI at SPI speed.
Okay so the port has to be the bus master, the MEGA2560 has 3 hardware serial ports to spare.
SD access will run quicker on a dedicated bus to MCU, not having to restart after re-select on a shared bus.