Programming Arduino Mega through serial Port different than RX0 and TX0

Yes it's possible. You will need an ISP programmer. If you don't have a dedicated programmer you can use an extra Arduino board as an "Arduino as ISP" programmer. I believe it's possible to use the ESP8266 for this purpose using ESP8266AVRISP (available via File > Examples > ESP8266AVRISP > Arduino_Wifi_AVRISP when you have an ESP8266 board selected from the Tools > Board menu) but you'd need to do some research on that because I have no experience with that.

Here's the easiest way to do it:

2560.menu.clock.16MHz_external.upload.port=UART0

to:

2560.menu.clock.16MHz_external.upload.port=UART1
  • (That is for uploading to Serial1. If you prefer a different port then adjust the above instructions accordingly.)
  • Save the file
  • Close all Arduino IDE windows
  • Restart the Arduino IDE
  • Connect the ISP programmer to your Mega.
  • Plug the ISP programmer in to your computer.
  • Tools > Programmer > select the appropriate selection for your programmer
  • Tools > Burn Bootloader

Note that you will no longer be able to upload to your Mega via the USB serial port (Serial0) after doing this. You can always repeat the above instructions to change the port or just select Tools > Board > Arduino/Genuino Mega or Mega 2560 and then do a Burn Bootloader.