how arduino bootloader work ?

Hello experts,
I am highly interested to know how the arduino bootloader works i.e. along with the IDE..
I read the AVR self programming i.e. the bootloader is the thing that can write the code in the application data section and so the chip gets burned without the ISP programmer because of the bootloader present in it.
I would like to know the exact action and steps taking place when one enters a command or clicks on the burn sketch in Arduino IDE..
The data is sent throught the serial connection as I guess, so how does the program data and the normal data is distinguished transmitting through the serial terminal of Arduino ??

I don't know the answer to your questions, but I think this is a good starting point:

STK500 protocol.

You have the bootloader sources. Read them.

You can turn on verbose logging of the upload in the Arduino preferences. That will log the step-by-step process of uploading.

You can read about the 'avrdude' program that does the uploading.

Yep, find Optiboot.c and read that. That will give you a good idea of what the bootloader is up to.

http://code.google.com/p/optiboot/

The "communications protocol" (such as it is) is described in Atmel App Note AVR061 (doc2525.pdf)
http://www.atmel.com/Images/doc2525.pdf

The data is sent throught the serial connection as I guess, so how does the program data and the normal data is distinguished transmitting through the serial terminal of Arduino ??

I think the short answer is the DTR pin of the serial connection is linked to the reset pin of the avr, so the IDE can reset the board before sending new code and so bootloader runs before already loaded code.

sorry for the late answer.