x50505:
I also have very little understanding of electronics, so if I am wrong, I would like to be corrected.The current will flow from the lower potential to the higher potential. Thus if the analog pin value is set to say 125 then it will have about +2.5V potential. The other side of the circuit is plugged into +5V socket. The current will flow from the analog side (+2.5V) through the LED and resistor to +5V. You are assuming that the ground means 0V at the pin. It can be so but it does not have to be the case. If you set the analog output pin to 0 instead of 125 the pin voltage will be set 0V. Note that in both cases (2.5V and 0V) the analog pins being set below 5V serve as ground.
You are confusing the electron charges with the potential. Without opening a physics debate, the current always flows from HIGH potential to LOW potential. If you want to know how/why you can search over the internet.
The current will flow between any 2 different potentials given the appropriate components are between. So having a resistor with 5.001 V on one side and 5.000 V on the other side will "cause" the current flow. There is a misunderstanding on how analogWrite() function works. If you set it to 125 it will not give you 2.5v on the pin. It will give you 0v for a very short period (1ms) and 5v for the same period (1ms). Therefore the current will flow for 50 % of the time. If you put a multimeter it will show you 2.5V; not because this is accurate, but because the multimeter cannot measure that fast and it makes an average.
An LED behaves just like a diode ( and thus the name, Light Emitting Diode). It allows current flow only in one side: from ANOD to CATOD. So if you have 1v to CATOD and 0V to ANOD it will not flow. If you want your LED to light, you need to provide a voltage on ANOD that is higher then on CATOD. How much higher ? Depends on the LED. Most of them have 0.6v threshold. So , for example, if you apply at least 1v to ANOD and 0.4v to CATOD it should work.
I hope it is a little bit more clear ...