There should be a place to put libraries that are not part of Arduino distribution. The current scheme makes you lose your libraries when you upgrade, or like me are foolish enough to blow away your library directory to update from the current builds. (Thank you backups.)
I don't know how things are laid out on other platforms, but something like ~/Documents/Arduino/Libraries/ would be convenient for Macs. Add it to the search list for libraries, exclude it from the sketch list. That way all my Arduino code would be in one spot.