I have looked over at teh multi-core thread, and my feeling is that it was basically "two arduino connect by I2C". Is there a way to modify it so that the Arduino can basically treat it as a dual-core CPU, such as bootloader? If not, how may I start to do so?
I'm not sure that is specific enough.
Each chip has it's own flash memory and processor and ram and eprom and I/O ports. The 168 chips do not share these unless you program some of their flash to do so in a pre-defined manner (i.e. a protocol over i2c or uart or...). But they can operate independently, and synchronize occasionally over i2c.
Ref: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1205243372
The issue currently, as I said, is that during programming, you have to program one core, flip a switch, then program another. What I want is that, at the very least, I can simply upload the sketches without doign switch flipping. Optimally, I want to be able to do some multi-thread programming with such setup.
And I am aware it does take some thinking, maybe even writing of bootloader, but hey, I am willing to try it.
I am aware it does take some thinking, maybe even writing of bootloader
I am pretty sure you'll need both.
And for making mutexes you sure need some means of communication:
...synchronize occasionally over i2c.
I think this i a complex and interesting project, and if you're up for it, why not give it a try? 
Several of Liquidware's shields have a 2nd AVR on the shield, and they allow you to write code for this 2nd AVR. So, when using these, you are running multi-cores (one core in you Arduino, and one core in each shield). Their products (shields) that have AVR's embedded in them are the TouchSheild Stealth, TouchShield Slide, and TankShield.
Once you start programming with these you immediately realize how complex working with multi-cores can become... which core is running that function? how shall i pass the data to the other core? Their demo apps give you some examples, but you can certainly re-configure them to work in any method you want.
Usually I use my Arduino as the main core and leverage the core on the shield as a "display control core". (which is kind of how the products are designed). It's kind of like how PCs have their own GPU.
It's something you may want to look at, before you develop your own board.
Cheers,
33W