I am getting different reading when using AnalogRead ( Around 0.5V difference ) when tested with a multimeter. Even when there is no input the output is around 0.7V. I am using an Arduino Mega.
How do I rectify this?
Don't expect any useful data from a floating (not connected to anything) input. It will be garbage. Analog inputs are referenced to ground so the ground of the voltage source that you want to measure must be connected to Arduino ground. Connect the analog input to ground and you should see very close to 0.
How do you power the Arduino?
does it have 5V on its 5V pin?
What you could do is use 2 analog pins.
A0 reads the MEGA 3.3V reference pin => you get a reference reading, call it ref33
should be 675 or 676 but if your 5V is not 5V it is different
A1 reads the voltage, with the following code.
float ref33 = analogRead(A0) /675.0;
float voltage = analogRead(A1) * 5.0 / 1023.0 * ref33;