Arduino Board Gets powered up from GPIO pin connected to another Arduino

Is this normal,

Yes

Will Device 2 get damaged due to this.

Yes
It is called parasitic powering and can cause latch up and damage the unpowered device.

what are the solutions to avoid this

  1. Do not apply active signals to an unpowered chip.

If you must then an opto isolator will prevent the parasitic powering but it will not work with analog signals. A cheaper but less effective solution is to have something like a 10K resistor in series with the signal lines.