Where are default arduino libraries located?

In particular, the GSM.h header files but in general Serial, etc...?

Take a look at:

If you mean where on your PC, they're in "\Sketchbook\libraries" or "\Program Files\Arduino\libraries" (GSM is here.)

Hardware Serial is here:- "\Program Files\Arduino\hardware\arduino\avr\cores\arduino"

The ones that aren't installed yet can often be accessed from the IDE, >Sketch >Include Library >Manage Libraries.