I also have very little understanding of electronics, so if I am wrong, I would like to be corrected.
The current will flow from the lower potential to the higher potential. Thus if the analog pin value is set to say 125 then it will have about +2.5V potential. The other side of the circuit is plugged into +5V socket. The current will flow from the analog side (+2.5V) through the LED and resistor to +5V. You are assuming that the ground means 0V at the pin. It can be so but it does not have to be the case. If you set the analog output pin to 0 instead of 125 the pin voltage will be set 0V. Note that in both cases (2.5V and 0V) the analog pins being set below 5V serve as ground.