If you haven't been using resistors with the LEDs , or tried to connect the motor directly to the Arduino pin, then yes you probably have messed up something.
There are lots of tutorials on the web for doing these kind of projects and the reasons behind the resistors and transistors used to drive things from seperate power supplies. Mostly it boils down to not drawing over the specified 40mA from any one pin, or 200mA combined from all.
There are ways to bypass the voltage regulator to test the AT328, but you run the risk of letting ALL the magic smoke out if you don't know what your doing, and I'm not about to be your accomplice in that.
This part of the forum will answer lots of questions about running high power stuff.http://arduino.cc/forum/index.php/board,9.0.html