It's a common mistake for people new to electronics.
I have some standard text I use about earth and 0V, it's not quite the answer to your question, which is why I didn't post it above, but I think you might find it useful. Here it is:
Earth and 0V
Earth (or ground) and 0V are often talked about as if they are the same thing. They are not, although, as they are often connected together, they can often be regarded as the same.
0V (zero volts, often incorrectly called “no volts”) is a point in a circuit designated by the circuit designer. 0V is the point to which all other voltages in the circuit are referenced unless otherwise stated. By definition 0V cannot be noisy. 0V is often but not always the most negative point in the circuit.
Earth is that muddy stuff outside. There is only one earth, if you are in any doubt, go outside and look around. In electrical terms when people say “earth” they really mean a connection to earth, often through the earth wire in the premises electrical distribution system.
An important thing to consider is that, while there is only one earth, there can be different connections to it and they are not all the same. In the UK (I don’t know about other countries) the earth connection in the building electrical distribution cabling is connected to the incoming neutral wire of the feed into the building before any fuse, meter or isolator. The neutral itself will be connected to the earth (the actual muddy stuff) at the distribution transformer. This means that an earth connection made to the building electrical earth might not be at the same voltage as an earth connection made to a metal stake pushed into the earth outside the building. As there is likely to be current flowing in the neutral back to the transformer then there will be a voltage drop across the neutral wire’s resistance. You can measure this voltage by putting a metal stake in the ground and measuring the voltage between the stake and the electrical system’s earth, most of the time this will be a few volts AC at most. However, if the transformer is hit by lighting then there is the possibility that the electrical system earth could be different from the earth outside the building by many tens or hundreds of thousands of volts, at least for a fraction of a second.
Something powered by batteries but not connected to anything else has a 0V but not an earth.
Many electronic devices, including Arduino, run off a power supply (the USB cable for example) that has its negative terminal connected to earth, so 0V and earth are connected and can be considered the same, and the supply is +5V (or whatever).
Many plug in mains adaptors do not have an earth connection (the earth pin is made of plastic), so anything running off them does not have an earth unless otherwise provided.
In my industry, telecoms, the positive pole of the supply is connected to earth, so the equipment runs on a -50V supply.
Analogue audio equipment typically runs off a split supply of -15V, 0V +15V, so it is the middle of the supply that is connected to earth, not the most negative part.