Interrupt/possible denounce problem

My Goal is to have a simple system move something left or right based on a button press. I have a button hooked up with a pull down resistor. I would like a double press (pressed twice within 5 seconds) causes a move right. A single with in 5 seconds moves left. Any press after that 5 seconds will stop the movement.

This is my first time using an interrupt and I think I have something fundamentally wrong. I can't seem to count my presses properly and sometimes it will just stop and seem to do nothing. I have tried to attach and detach the interrupts at different times as well as adding delays to give it a debounce.

Thanks for the help

int count=0;
unsigned long time;

void setup()
{
  pinMode(pin, OUTPUT);
  attachInterrupt(0, btpress, RISING);
  Serial.begin(9600);
}

void loop()
{
  if(count == 1 && millis() >= time)
  {
    Serial.println("move left");
    delay(500);
  }
  if(count == 2 && millis() >= time)
  {
    Serial.println("move right");
    delay(500);
  }
}

void btpress()
{
  if(count == 1 && millis() <= time)
  {
     count=2;
     loop();
  }

  if(count == 1 && millis() >= time)
  {
     count=0;
     Serial.println("turning off left");
     loop();
  }
    
  if(count == 0)
  {
    count = 1;
    Serial.println("pressed once");
    time=millis()+5000;
    loop();
  }
  
   if(count == 2 && millis() >= time)
  {
     count=0;
     Serial.println("turning off right");
  }
}
void btpress()
{
  if(count == 1 && millis() <= time)
  {
     count=2;
     loop();
  }

Don't call loop.

void btpress()
{
...
  if(count == 1 && millis() >= time)
  {
     count=0;
     Serial.println("turning off left");
     loop();
  }

Don't do serial prints inside an ISR.

This is my first time using an interrupt and I think I have something fundamentally wrong.

It's great to learn the use of interrupts, however using them to detect button presses is usually redundant. The main loop iteration speed is normally orders of magnitude faster than human reaction time for pressing a button.