In some neighborhoods it will be very safe, I have tested this theory with other valuables.
The Adafruit Ultimate GPS will work well in the woods. Others will not.
I'm relying on the good nature of cachers to prevent the device from disappearing, but just in case, I have also written into the program a routine that, when it gets the satellite lock, verifies it's within the boundaries of the forest preserve where the cache lives; if it detects it's outside the bounds it will bring up a message "please return me to ..." and go comatose.
I went with the GPS from SparkFun; it's been pretty accurate and fast to lock on so far. I'm taking it on vacation next week in all its breadboard glory and verify that it will work in dense tree cover; if the results aren't great I'll look into the Adafruit one.