But still I would like to upload some code to my Arduino, is it safe to plug in usb cable at the same time with dc adaptor?
Yes, provided the DC adaptor is within the on-board voltage regulator's input range. Arduinos with an on-board voltage regulator are designed to always take power from the regulator when available, even when connected to a computer via USB. In that case only the USB cable is only used for serial communication.
In more case, is it safe to plug in usb cable, dc adaptor, and 4 AA battery at the same time?
If the DC adaptor is pulgged into the barrel jack and the AA battery pack is correctly connected to VIN and GND pins, it won't cause any harm... However, you wouldn't want to do that without at least putting a diode in between the positive lead from the battery pack and the VIN pin. This will make sure the current is only being drawn from AA batteries when the DC adaptor is unplugged or otherwise not providing current.