As suggested, I think that you could use the ESP32-C3-WROOM module from Expressiff instead of the bare chip, it will make your life a lot easier and reduce the risk of bad performance. The antenna, xtal and the rest is in the module and properly designed, and it's small and compact.
Check in the datasheet, section 9 "Peripheral Schematics" for the correct placement of the rest of components. I the section 8 you can also see what is inside.
And read carefully all the recommendations in the datasheet. Don't put any component or GND plane below the antenna. But I would fill the bottom layer with a solid GND plane and remove everything what is possible from there.
How do you plan to program it? You could use D+, D- from the usb connector, as suggested in the datasheet. Then you woudn't need a TTL adapter.
Regarding power consumption, the AMS1117 has a quiescent current of 5-11 mA, what is too much for sleep mode. You could find another LDO with quiescent current below 1mA, or it will drain the battery faster.
And the p-mosfet U5 to disconnect the battery when the USB is connected, is correct. But I think that you should tie the gate to GND with a resistor, ~ 10KΩ. Otherwise when the USB is not connected, the gate would be floating and maybe the mosfet is not fully conducting.
EDIT: I see that it's already there.
And the gate resistor, R5, could be smaller 200Ω or 300Ω.