Averaged GPS altitude cushioned by pressure readings passed through a Kalman filter should give a reasonable estimate when weather conditions are stable. Tuning the filter right however is challenging as weather is such a dynamic and unpredictable process. A number of handheld GPS units (some models by Garmin at least) include a barometric pressure sensor for this purpose.
Altimeters on airplanes however use only a barometric sensor. This is what is used to maintain vertical separation between airplanes and so is based on relative measurements against a common reference (established through a pre-flight calibration procedure).
If you proceed to implement this, it would be interesting to see some facts, figures and conclusions.