I understand Arudino consists of both the software (ide + wiring libraries) and hardware (avr, specific header pinout)..
But I'd like to design my own pcb... with some robot specific features. I plan to use the new open hardware license, and of course I won't be using the "arduino" brand on anything, but I'm curious what exactly does it take in order for full compatibility?
For example, in the IDE, I would love for my would-be board to be listed. I realize I'd have to add to the software libraries for people to access the added functionality, but is that even possible, or would I have to branch completely on my own... ala 'Freeduino Epic'?
With so many flavors of official arduino boards out there, I assume they've made it easy for themselves to port the ide/software to new hardware, but can I add support for a non-"Arudino" brand board?
If you stick to the supported processors you don't have to do anything software-wise. Just build it and it will work.
If you have non-standard hardware you'll have to modify some of the core files unless you are happy to place a caveat on the board, ie "A0 is used to monitor the VIN level, don't use it in your program" .
You can also add you stuff to the IDE and have it appear on the menu, but I haven't done that so I'll leave it for someone who has.
The easiest way to see the required changes is to look in the /hardware/ folder inside your main Arduino installation. This set of files defines a board as far as the IDE is concerned. In recent versions, it'll also look for board definitions like this inside a 'hardware' folder in the user's sketch directory. For the most part, you can copy the /hardware/arduino folder there, change its name and modify a few files to support your board.
Files to edit:
'boards.txt' - tells Arduino IDE how to list your board in the IDE, and some other information e.g. clock speed, bootloader protocol/speed and amount of Flash.
/cores/arduino/pins_arduino.h and pins_arduino.c: If your board has changed the pin map from a standard Arduino, there is a set of structures here that implements the pin numberings used by digitalWrite(), etc. that should be updated.
With this modified clone of the /hardware/arduino folder, you can zip it up and have users unzip it into the 'hardware' folder in their sketch folder (create if necessary), and the new board will appear next time the IDE is restarted.
Likewise with custom libraries - distribute custom libraries as a folder that can be dropped into the 'libraries' folder in the user's sketch folder (again, create if necessary). For better or worse, as far as I know all installed libraries will appear regardless of the selected board, so there is no way to have a 'board-specific' library that only appears for a particular board.