The use of angle brackets (<>) causes the compiler to look for the file on the default path for the library directory. Depending upon installation, this path is the Documents/Arduino/libraries directory on a Windows machine.
The use of double quotes ("") around the header file causes the compiler to look in the sketch working directory for the header file. The default sketch directory can be seen in the File --> Preferences menu option. If the file is not found there, it will also search the default library directory.