Does mega2560 require pullup resistors on rxd0 and txd0?

I made a mega2560 minimum system, with only the chip, resonator, i2c ports, serial ports and power pins out. When I uploaded a sketch using a usb-ttl adapter (the very ordinary one), it works fine, and the sketch is execuated. But when I let the system run alone by itself by powering it up with a 3.3v pins, without any serial pins connected, the chip seems to not bootable. I have the ordinary bootloader burned already and uploaded the sketch to the chip using a regular serial0 way.

Then I checked usb-ttl adapter and found that the rx and tx pins seem to have pull up resistors. I do remember that some old mcus don't have internal pull up resistors for their serial ports. Is this the case for 2560?

When the system runs, it dosen't need the rx0 or tx0. I run the system alone, with a pull up resistor on rx0 and it works...

I checked the arduino mega 2560 schematic and didn't find pull up resistors on their serial ports. Or maybe the usb-ttl chip has those inside? Does fuse bits do anything on this pull up behavior?

Or maybe, when I pull the rx0 up manually, it happened to screen the board better, so that the system started to run?

If the RX pin is floating, the bootloader might get confused. You could add a 10k pullup resistor to RX.
I think that your main problem is that the ATmega2560 does not run well at 3.3V at 16MHz. It needs at least 4.5V.
The ATmega2560 is the oldest chip that Arduino has, and it is the least tolerant to lower voltages. Maybe the chip production has improved, but it is better to use it according to the datasheet.

Manufacturer page of the ATmega2560: https://www.microchip.com/wwwproducts/en/ATmega2560 with datasheet.

oh. That makes sense. So on an arduino board, ide send reset signal by using tx and rx, so leaving them floating confused the bootloader.

After a power on or after a reset, the bootloader runs first. The bootloader checks the serial port to see if it has to do something. If there is nothing at the serial port, then it starts the sketch.

When the Arduino IDE wants to upload a sketch, it sends a pulse with the DTR signal which is connected to the RESET pin via a capacitor. That resets the Arduino and then the Arduino IDE can try to communicate with the bootloader.

Other boards have other procedures. This is for the basic boards such as Uno or Mega.

However, nothing makes sense anymore if you run a 16MHz ATmega2560 at 3.3V. It needs 4.5V, even 4.4V is too low.

If a lower operating frequency is acceptable, ATmega2560 can operate at 3.3 volts. In datasheet on page 358 for 8MHz is needed Vcc 2.7V.

Do you need an 8MHz clock instead of 16 for that operation?

Use the prescaler value 2 to divide the clock to 8MHz.
It can be changed by software.
But if the xtal is 16MHz, the rating can't be met for a very short time after the power on until the prescaler is set.
So use the CKDIV8 fuse to set the default prescaler setting to 8.
After that fuse is set, uC start at 2MHz at 16MHz xtal.
Immediately after booting, the prescaler can be changed to 2 (8MHz).
In which case there will be nothing time to running by exceed the rating.

Note:
Probably you may need to modify your bootloader to compatible with prescaler...