John,
You put the LED and current limiting resistor between the Arduino pin and ground,, and the LED 'Sinks' (draws) power from the Arduino when the pin goes high ( the Arduino sources power to the LED).
You put the LED and resistor between +5Vdc and the Arduino pin,, and the LED 'Sources' power, from +5Vdc, to the Arduino pin when it goes low ( the Arduino sinks from the LED).
I think,,, My brain hurts....