So I'm thinking of making a USB microphone using my arduino (or perhaps I'll settle for a smaller atmel chip later), but I have no idea of where to start when it comes to coding this. The analog part of it that is amplifying the signal from the electret mic and then feeding it to the analog input of the atmega is easy, the issue is with USB. I've tried to read a little bit about this protocol, but It hasn't made me a wiser man, all I'm hearing is people saying that there's more important things in the world than to learn the USB protocol (I assume it's complex?). I know there are MCU's with a USB peripheral, but these are more costly and bigger. It would be a lot neater if I could use an attiny and bitbang the USB instead 
But I really need some guidance as I have no idea of where to start 
but I have no idea of where to start when it comes to coding this.
First, you need some requirements. What does the program need to do?
More importantly, what the heck is a USB microphone?
I assume it's complex?
Is that a question or a statement? If it's a question, I don't know how you expect us to know if you think it's complex. If it's a statement, I agree, even though the punctuation is wrong.
I know there are MCU's with a USB peripheral, but these are more costly
Gee, I wonder why.
It would be a lot neater if I could use an attiny and bitbang the USB instead
USB involves hardware. The ATTiny really isn't going to make up for the lack of hardware by bit banging anything.
But I really need some guidance as I have no idea of where to start
Project guidance is where. Although, really, realistic expectations come first.
Not heard of a USB microphone? Don't worry, its rather an easy concept to grasp!
Most small home-recording setups use them now, usually a reasonable/excellent quality
condenser mic + preamp + ADC that pipes 16 or 24 bit 48/96kSPS digital audio down USB at your favorite music software.
Cheaper versions are available as part of a USB headset.
But USB isn't a simple protocol (designed by a committee, you can
always tell). You'd start with a microcontroller that supports USB
in hardware like Leonardo or Due.
...all I'm hearing is people saying that there's more important things in the world than to learn the USB protocol (I assume it's complex?).
That depends on what you are doing... If you are writing the firmware for a USB device (like a microphone) it's kind-of important. 
The good news is that Widows & OS X (and I assume Linux) have standard drivers for USB soundcards & audio devices (and a few other things like USB keyboards & mice) , so you don't have to write a driver on the computer-side. If you are creating a more specialized device (or something like a printer), then you have to write the driver and the device firmware.
The bad news is that you have to understand how to communicate with the drivers and develop your firmware accordingly (there's a bit more to it than just understanding the basic USB protocol). I'm sure you can the Widows details somewhere on [u]MSDN[/u], and I'm sure it aint' easy!!!!
You'll notice that even the Arduino IDE developers didn't bother with true USB... They used FTDI USB Drivers so the IDE "thinks" it's communicating over an RS-232 COM port.
So, I'd say it's a BIG PROJECT. Probably a good learning experience, but otherwise not worth it unless you want to build something that you can't buy.
Thanks for the replies. The idea is to learn about USB, but if it's really that much of a hassle then I might consider learning something else instead Aren't there anyone who has already written the code to bit banging a USB sound card though?
Plecto:
Aren't there anyone who has already written the code to bit banging a USB sound card though?
USB is a complex protocol that doesn't work with "bit banging."
The whole point of USB was "keep the hardware simple and move as much to software as posssible", which should answer your question.
There is a tutorial an the site here that describes how to program the USB chip on a UNO to be a midi device instead of a serial device, that might start you in a direction.
I was thinking that USB could be the next step for me. I'm looking for something new to learn, but I don't really know what would be appropriate. I'm quite comfortable with serial communication (using the uart, SPI and I2C to communicate with other ICs) and using LSD displays is no problem either. What would be the appropriate next step if not USB?
You might read up on the open source V-USB project, which does low speed USB bit banging on the smallest AVR micros.
There is also the USB-tiny project USBtiny