Millis (); timer not operating as a delay command. help?

my project is simple. I am basically building my own thermostat device. I want to turn on a heater fan at a set temperature, according to a temperature sensor. Simple enough, but I don't want the fan to flip on and off every 30 seconds as the temperature changes. My original code consisted of:

if (temp <= setTemp) //Turn fan on if room temp is lower than set temp.
{
  digitalWrite(outputPin, HIGH);
}
else //turn fan off if temp is 
   {
     digitalWrite(outputPin, LOW);
     delay(5000); // keeps fan off for a set time so that the relay won't flip off and on quickly.
     }

However, as many of you already know, a "delay (5000);" command simply freezes the entire loop. This command easily completes my goal, but I cannot adjust my set temperature while the fan is off. This is called "blocking." I decided I would try unblocking by using a timer according to this method: Arduino Playground - AvoidDelay
however, my outputPin does not stay turned off for my set time. Here is my new code. Can anyone tell me why my delay does not work? NOTE: THIS IS NOT MY FULL CODE, ONLY THE TIMER BIT. IF REQUESTED, I CAN POST THE WHOLE CODE, BUT I'M RUNNING OUT OF ROOM. THANKS IN ADVANCE!
Code with Timer:

#include "Timer.h" 
Timer s;

void setup() 
{ 
  pinMode(outputPin, OUTPUT);
   s.pulse (outputPin, 10 * 60 * 1000, LOW); //turn off for 10 minutes
}

if(temp <= setTemp) 
{
  digitalWrite(outputPin, HIGH);
}
else
{
     digitalWrite(outputPin, LOW);
     s.update();
     //delay(5000);
}

newtime = millis()
if newtime - oldtime <= settime then no change possible
else
oldtime=newtime
change possible

for each timer make separate vars to not influencing oneanother

One common solution to this is simply to have a dead zone around the set point. Turn the fan on when the temp is low and outside the dead zone. Turn it off when it is out of the dead zone on the high end.

@Shooter: Can you show me that in an applicable code format with my code? I'm still very much so a newbie and I'm not good and implementing pseudo codes. Thanks!

@wildbill I've tried that with a code like: (Just for example) if above 75 turn off, if over 80 degrees turn off and to create a "window" of function, but it still bounces off and on around 80 and 79 and then 74 and 75. Can you give me a code example of how to do that?

const byte outputPin=5;
const int  Setpoint = 77;
const int  TempLeeway = 3;
int Temperature;

void setup() 
{ 
pinMode(outputPin, OUTPUT);
}

void loop()
{
Temperature=ReadTemperatureBySomeMeans();
if(Temperature < Setpoint-TempLeeway) 
  {
  digitalWrite(outputPin, HIGH);
  }
if(Temperature > Setpoint+TempLeeway) 
  {
  digitalWrite(outputPin, LOW);
  }
}

@wildbill Thanks for that code! I will try it and report back asap

@wildbill Thanks so much for that code! It works exactly how I need it too. XD What kills me is that I had this idea a few weeks ago and I must've been mixing up the order somehow. Thanks again!