I'm on the same page, I just started to program AVRs but I'm after a solution where I could "remotely" update an AtTiny/AtMega AVR.
One approach would be to store the new AVR's Image in the EEPROM and make a bootloader capable of self programming it reinaldoaf is proposing.
Another option is to use a secondary AVR chip on the same board. This approach would add $3-$5 in components but might be safer... if something happen in the middle of firmware upgrade the process could simply be restarted and the "programmer" AVR would re-try.
At the moment this project was the closest I found to use an AVR to program another AVR: GitHub - adafruit/Standalone-Arduino-AVR-ISP-programmer: A standalone programmer for mass-programming AVR chips
Should be OK to replace the internal EEPROM with an I2C EEPROM and use a smaller AVR (AtTiny?!)