Each different LED process has a different forward voltage drop, and each LED kind will typically have a different target current, too.
Thus, a low-current red LED with 1.8V forward voltage drop and 2 mA design current would need a 1.6 kOhm resistor from a 5V output ((5.0-1.
/0.002.) A medium-current green LED with 2.1V forward voltage drop and 10 mA design current would need a 290 Ohm resistor ((5.0-2.1)/0.010.) A high-current blue LED with 3.0V forward voltage and 25 mA design current would need a 80 Ohm resistor ((5.0-3.0)/0.025.)
You can find the forward voltage and current needs for your LED by reading the data sheet.
However, it turns out that most LEDs will do fine with significant over- and under-currents. Thus, a 220 Ohm or 470 Ohm resistor or even a 1.0 kOhm resistor may be "good enough" for most LEDs to make some light and not immediately burn out. Thus, many tutorials will recommend one of these resistors without going into how that value was derived.
When it comes to running a common anode LED, that's not that different from anything else. As you noticed, driving the pin low will make the AVR CPU sink current, and will turn the LED on, if the LED anode is at +5V. It may also burn out the LED if you don't have a current limiting resistor, and may even burn out the Arduino if the LED current consumption is too high. The Arduino is acting as a "low side switch" here, which is not particularly uncommon in modern electronics.
However, your description of driving the pin high is not accurate. Setting the output to +5V on one end, and connecting the anode to +5V from the other end, will not "drive the LED from both sides." Any component only sees the voltage differential between its sides, and when both sides are at +5V, the differential is 0V, and there will be no current flowing. Thus is the same thing as tying both ends to ground -- the differential would be 0V, and no current would flow.
To close the loop here, a "high" output pin from an Arduino is actually more like 4.75V-4.8V, because of losses in the transistors that turn the ports on/off and other internal circuitry in the AVR controller. Thus, if you tie the anode to +5V, and drive the other pin high in an output, you will see a voltage differential of 0.25V. However, this is sufficiently below the forward voltage drop of the diode, that effectively no current will flow, and the diode will not turn on. Even if you replaced the diode with a resistor, it would be OK, because the AVR can take a slight overvolt on output pins, as long as the current isn't higher than specified (25 mA typical, 40 mA max.) It's a very robust microcontroller -- don't expect every chip to be that nice! The Arduino Due is reported to be much less robust (and reading the data sheet, that would make sense.)