With our commercial Power equipment we recently had a situation where there was 30VAC between neutral and earth. Of course my 3 phase systems barked at that and threw a few sparks - we turned it off.
The culprit was likely a large 3 phase motor - as the earth leakage from active manifested across each phase but each phase had a slightly different earth leakage.
My advice would be to take a common 'ground' from the low power side - that is, from the circuit supplying the analog AC signal.
Similarly when connecting a microcontroller to a sensor there must be a common reference ie a negative of the dc power supplying the circuit. You can common the circuit negative to all board power ie 5Vdc, 12Vdc etc - include the sensor negative eg the opamp -ve and it will work.
Of course you will need to make sure that the analog signal is in the 0-5V range. Suggest using a voltage divider and place the signal somewhere in the middle of the range, then you need to ensure that any attenuation in the signal falls within the voltage spec. to the analog input - easy to do - use a CRO to check signal levels as rms before connecting to the Arduino. Suggest using a 100:1 probe to inhibit circuit impedance problems. 10:1 probes and lower will add impedance and corrupt the signal flow, so when you disconnect the CRO your calibrations will be off.
Good luck.