I've connected GPS, accelerometer, magnetometer, and temperature sensor to Arduino. See http://www.zbotic.com/index.php/download/ for links to the devices and a library. The Arduino can easily connect those devices and read from them. You could then send their output to Arduino USB/Serial monitor on a PC, or save the data to an SD card on the Arduino.
Unfortunately there are a bunch of different types of connectors for sensors and they have Ground, Signal, and +V on different pins. I'm pretty careful but I've managed to melt (literally) three sensors by connecting the wrong polarity - by some miracle my Arduino and all the ports are still working. You should consider www.seeedstudio.com and use their "Grove" devices. Grove is a standard set of connectors for sensors and Seeedstudio is pretty cheap (although if you're not in Asia, shipping is slow). Grove is also available from some sites in North Am (and probably Europe). Personnally I'm not a fan of tinkerkit - they have a smaller number of sensors and their pinout seems the least popular.
You' may want to get an I/O shield (gives you a bunch of connectors for the sensors). I2C is good for the more complicated sensors since it's a bus and allows complex communications and multiple devices. For I2C you might want http://www.seeedstudio.com/depot/grove-i2c-hub-p-851.html?cPath=178_191 since it let's you connect multiple I2C devices (using grove connectors).