Although I personally dislike the use of relays, this would probably be a situation where you'd want to use one. You'll definitely want to upsize the current on it, as if it has any kind of motor on there you'll end up with a current spike every time it turns out. Just a basic search on DigiKey led me to this guy. At $1.50, it will handle up to 10A at 250VAC or at 24VDC.
Activating the relay is as simple as applying 5VDC across the coil and the other side will activate, allowing your heater to turn on.
The thermistor will work to measure temperature but typically I've seen them used as a crude way to limit current in an application. What you would need to do to get a thermistor to work is create a voltage divider and read the voltage across the thermistor. Once you have the voltage, do some math to determine the resistance, and then use the data sheet to map that resistance to a temperature, at which point you can make a decision regarding the heater.
You may be better off with a thermocouple, as instead of changing resistance due to temperature, it will create a very small voltage. Running this voltage through an op-amp would give you a value you could read and again, make a decision regarding the heater.
The PID control that the other poster mentioned refers to a Proportional Integral Derivative controller. While there's nothing inherently wrong with measuring a temperature, turning the heater on if it's too cold, and off if it's too warm, you will run in to issues where you basically turn off and on all of the time to maintain that temperature, and you'll forever oscillate around your desired temperature. With a relay being a mechanical device, you'll eventually burn it out(not for a long long time, but it will burn out eventually).
PID control is essentially a way of using calculus to dampen that oscillation and allow the temperature controller to behave less erratically. Unfortunately, I've seen it utilized in LabVIEW before, however I'm not quite sure off of the top of my head to perform an integral or derivative in Arduino C(I'm sure it's possible, it's just too late for me to be trying to think that through). You can read more about it here, and maybe somebody out there has done PID with arduino before, I'm just not able to say one way or another.
Long story short, it all depends on you and the scope of your project. If it's something where you're trying to warm your van or whatever in the winter, I'd say throw a relay and thermistor on it and forget any kind of fancy control system and call it a day. If you're trying to incubate a chicken egg(or whatever) and accurate temperature is more of a necessity, well of a sudden the straight on off system doesn't look so good, and you'll want something slightly more accurate than a thermistor.