Rotary Encoder Help

I'm working on a project for a dummy load and I'm using a rotary encoder to adjust the size of the load. Here's a section of my code:

#include <SoftwareSerial.h>
#include <serLCD.h>

serLCD lcd(4);
enum PinAssignments {
  encoderPinA = 2,   // rigth
  encoderPinB = 3,   // left
};
volatile  int encoderPos[] = {0,0,0,0};  // a counter for the dial
int lastReportedPos[] = {1,1,1};   // change management
static boolean rotating=false;      // debounce management

// interrupt service routine vars
boolean A_set = false;              
boolean B_set = false;

void setup()
{
  
  pinMode(encoderPinA, INPUT); 
  pinMode(encoderPinB, INPUT);
  pinMode(7, INPUT); 
  
 // turn on pullup resistors
  digitalWrite(encoderPinA, HIGH);
  digitalWrite(encoderPinB, HIGH);

// encoder pin on interrupt 0 (pin 2)
  attachInterrupt(0, doEncoderA, CHANGE);
// encoder pin on interrupt 1 (pin 3)
  attachInterrupt(1, doEncoderB, CHANGE);
}

void loop()
{
 Encoder();
 Power();
}


int Power()
{
 int P;
 P=encoderPos[0];
 lcd.setCursor(1,1);
 lcd.print("     Power      ");
 lcd.setCursor(2,1);
 lcd.print("P = ");
 lcd.setCursor(2,5);
 lcd.print(P);
 lcd.print("     ");
 
}

int Encoder()
{
 
  rotating = true;  // reset the debouncer
  
  if (lastReportedPos[0] != encoderPos[0]) {
    if(encoderPos[0]<0)encoderPos[0]=0;
     
    lastReportedPos[0] = encoderPos[0];   
  }
}

void doEncoderA(){
  // debounce
  if ( rotating ) delay (10);  // wait a little until the bouncing is done

  // Test transition, did things really change? 
  if( digitalRead(encoderPinA) != A_set ) {  // debounce once more
    A_set = !A_set;

    // adjust counter + if A leads B
    if ( A_set && !B_set )
    {
      encoderPos[0]++;

    }
    rotating = false;  // no more debouncing until loop() hits again
  }
}

// Interrupt on B changing state, same as A above
void doEncoderB(){
  if ( rotating ) delay (10);
  if( digitalRead(encoderPinB) != B_set ) {
    B_set = !B_set;
    //  adjust counter - 1 if B leads A
    if( B_set && !A_set )
   { 
      encoderPos[0]--;

   }

    rotating = false;
  }
}

The problem I keep running into is I will turn the knob one click to increment by one and sometimes it will work. Other times though, when I turn it one click it will increment by 4 or 3 OR it will decrement, sometimes by one, other times by two or three.

I'm not very familiar with rotary encoders so I'm struggling to debug the issue, I thought by using interrupts it would solve this problem for me but it hasn't.

If it helps, the encoder I'm using is the RGB Sparkfun encoder:Rotary Encoder - Illuminated (RGB) - COM-15141 - SparkFun Electronics

If someone can help me figure out what the problem is, I'd greatly appreciate it.

I think the hardware debounce solved my problem (along with removing the delay) thank you!

Another thing I learned is to use an ultra-high quality optical encoder without detents, not a junky mechanical one. One like this:

http://www.ebay.com/itm/370592502534

Damn, these are nice.