ATmega644p reboots on serial input

  1. You are running out of spec. The Speed Grades graph shows that 3.78V is needed to run & be fully within spec at 16 MHz.

You solved the OP's problem, that's all .