Library Problem With IDE V1.5.1r2 And Due

If I eliminate the one in {sketchbook_folder}\libraries, will the compiler find the right one in sam or avr according to board selection. Will this damage my ability to use V1.0.3 for other boards.