Ok, this seems to be a very common problem and I have not found a good resource for a solution so I'm hoping by compiling all my research and with the help of the following posts we can make some sort of guide for future users. I noticed similar post whose title describes my issue but doesn't explain much here ( http://arduino.cc/forum/index.php/topic,82085.0.html )
Scenario: People like programming in the Arduino / Wiring language. There are built in libraries and users don't have to worry about all the details of setting up timers and other non-trivial tasks. Also, being able to abstract individual pins from ports is nice way to think for newcomers. So for people not worried about maximum efficiency this style of coding is very nice; however, they still recognize that it's very wasteful to keep buying a whole new Arduino board for each project. They want to code in the Arduino language and then port that code to a basic ATmega328 or equivalent chip on their own custom board, but most are unsure how to do this including myself.
Problem Statement: People do not know how to get the 'fancy' Arduino code onto a generic ATmega328 chip.
In my case I thought "ok it's time to step up to the big leagues and get an ISP programmer so I can be a real microcontroller dev. I chose the AVR MKII and got AVR Studio 5 installed and had a Blink sketch from Sparkfun all working and was ready to go. I quickly found out that it is not simple to port your Arduino project to an AVR Studio project. All of the core files in Arduino which make development easy do not exist by default in a basic AVR project and including the whole arduino directory seemed like a mess so here I am wondering how I can get code that I know works on an UNO/Deumillenove and get it on a 328P using MKII.
Now this problem causes people to be opened up to whole set of other problems. I'm taking step back away from my personal story a bit to maybe help some others who have a slightly different case.
There are two most common ways to get code onto a microcontroller: 1. Through standard USB to Serial converter ( This is standard on Arduino boards ) 2. Using AVR ISP MKII
Background knowledge about each transfer method: 1. Using USB-Serial This involves extra hardware on the custom board and is less common for barebones microcontroller programming or custom projects which don't need USB. As far as I know this would work ok with the Arduino IDE however you just have to make sure you setup the Auto-reset correctly otherwise you will always have to reset your board just before uploading. ( I found this out the hard way when trying to use Eclipse to upload code. Even when using an actual UNO, Eclipse does not have the reset functionality built in like Arduino IDE has and you have to manually press the reset button which is quite annoying. )
2. Using AVR ISP MKII This is my particular case and although I did a bit more research about this method I still can not get it to work. Most people who have the AVR ISP MKII also have AVR Studio installed and this brings up another possible problem. Documented briefly here: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1228502992 AVR Studio has it's own utility / Jungo driver to work with the ISP MKII and Arduino IDE uses avrdude and libusb and since only one of these drivers can be installed at one time you either have to only use the MKII with AVR Studio or only use it with Arduino IDE.
So within this method of programming we have two more possibilities of Arduino IDE or AVR Studio.
Method 2.a ( Arduino IDE and AVR MK II ): If you're using the Arduino IDE with avrdude / libusb then you should just be able to select Upload with Programmer and it should work. You might want to look at this: http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1275160256 which explains how to modify the boards.txt and add another menu with custom settings for programming 328P's with MKII however when I did this I still didn't change the drivers and avrdude could not recognize my MKII usb device so I can't verify either of these solutions, but I believe uninstalling the Jungo driver from AVR Studio and installing the one that comes with the Arduino package would work.
Method 2.b ( AVR Studio with AVR MK II ): I was following this article http://arduino.cc/playground/Learning/Burn168 to use AVR Studio however I do not think it has the whole story. Once you upload the Atmega328 bootloader .hex ( ..\arduino-1.0\hardware\arduino\bootloaders\atmega\ATmegaBOOT_168_atmega328.hex ) and set all the fuses ( 0xF8, 0xDF, 0xFF ) you still need a way to upload an actual program / sketch .hex and this is where I was confused.
From my understanding once you upload the bootloader hex you then change the fuses to control where the memory offset will be for the next hex file so that when you try to upload a .hex from an Arduino sketch it will not over write the bootloader since you've told the microcontroler to skip over that memory region with the fuses.
In summary I tried the following with no success:
1. Developed on Arduino IDE on UNO R2.
Then from my understanding the Arduino IDE goes through the build process ( http://arduino.cc/en/Hacking/BuildProcess ) does a bunch of behind the scene's work to create a .hex file which is basically your sketch with all the includes from the core compiled, linked, and converted down to machine code. * From my understanding this file should be able to run on any similar microcontrollyer namely, the 328P which i'm tyring to use. ) So I just need to get this file upload to the chip.
2. Get the .hex file generated from Arduino IDE and copy it to sketch folder. Apparently the Arduino IDE does a lot of file manipulation under operation and in order to get the .hex file you need to open your sketch and click verify, but DO NOT click save or close the IDE because this will delete it from the temporary location. The .hex file generated will be in %localappdata%/Temp/[sketch_name].hex . ( The %localappdata% is a system variable which goes to the proper directory for Windows 7, it may be in some other Local App Data directory in older versions of windows. ) So I copy this to my sketch folder an then close the Arduino IDE.
3. Open AVR Studio 4. Upload the Arduino Bootloader, set Fuses, and set Lock Bits Again I used ( ..\arduino-1.0\hardware\arduino\bootloaders\atmega\ATmegaBOOT_168_atmega328.hex ) and set fuses and lock bits based off of the 168 section of this article http://arduino.cc/playground/Learning/Burn168
5. Upload the .hex file we copied to our sketch folder. In the same window we jsut used to program the bootloader we just browse to the new .hex file and click "Program"
However these steps didn't work for me and so I have the following questions:
1. Is it correct to assume that the .hex file generated by Arduino IDE can be used on a basic 328P?
2. If I do want to use this .hex file from Arduino to a blank 328P is a Arduino bootloader required to make the .hex file work or should the .hex operate correctly without the bootloader. From my understanding the bootloader was only there to retain serial communication regardless of what program is on the device. If we are using the ISP MKII we don't need USB communication and hence the bootloader so the .hex file form Arduino IDE should work independently. We would just have a little be extra flash memory to work with.
However, I read in another source that the bootloader also sets up timers and other things that are expected to be initialized a certain way by Arduino code so if they are not then perhaps PWM, Tones, and other operations would not function correclty without the bootloader. Hopefully someone can clarify this for people and myself.
3. If we use AVR Studio to program the bootloader should we then uncheck 'Erase device before upload' to upload the next .hex or do the Fuse/Lockbits already restrict what will be erased. My concern is that perhaps the .hex does require the bootloader and that even though the directions I followed were correct, I immediately erased the bootloader before uploading the new 'sketch' .hex and this is why I see no improvment from just uploading the .hex without the bootloader.
4. For someone who has used both methods ( Arduino IDE / ISP MKII OR AVR Studio / ISP MKII ) if I want to write code in the Wiring language but then upload to a blank 328P which is the best method in your opinion?
Sorry for the very long post that probably has an easy answer. I just wanted to be a complete as possible so people dn't have to ask for more information. Also if I mis-stated something let me know and I can edit it for future reference.
Thank you very much!