Libraries provide your sketches with extra functionality (e.g. the ability to control an LED matrix, or read an encoder, etc.). They were introduced in Arduino 0004.
To install a library, place it in its own folder inside ARDUINO/lib/targets/libraries. It will then appear in the Sketch | Import Library menu in the Arduino IDE. To use a library in a sketch, select it from that menu while editing the sketch. This will insert at the top of the sketch an #include statement for each header file (ending in .h) in the library's folder. These statements make available to your sketch the functions and constants defined by the library. They also signal the Arduino environment to link that library's code (contained in files ending with .c or .cpp) with your sketch it is compiled or uploaded.
Because libraries are uploaded to the board with your sketch, they increase the amount of space used on the ATmega8 on the board. If a sketch no longer needs a library, simply delete its #include statements from the top of your code. This will stop the Arduino IDE from linking the library with your sketch and decrease the amount of space used on the Arduino board.
To get started writing libraries, download this test library (http://www.arduino.cc/en/uploads/Main/Test.zip). It should provide a basic guide for creating a new library.