What are the values of the resistors.
Or.
Do the inputs of the opamp stay within the common mode range of the opamp.
That is about 3volt max on a +5volt opamp supply.
I guess not.
Try using an opamp that is designed for high-side sensing, like the INA167, INA196, INA219, INA 226 and many others.
Leo..