Arduino Tiny library with AVR

Can I do this and still program it with the arduino IDE

Yes it is pretty easy, maybe you have to make a new entry in the programmer.txt
I have succesfully used both USBasp programmer and Pololu programmer with the Arduino IDE take a look at this for example.

But that was until I learned about TinyISP

http://arduino.cc/forum/index.php/topic,123388.30.html

which makes debugging much easyer