A strange result when changing OCR1A in the interrupt

Sorry for my pool English. I want to gradually increase the interval between each Timer1_CTC interrupt routine so I change the value of OCR1A in the interrupt routine by the following codes:

#define BITDEP 5
Volatile uint8_t Counter;
uint16_t ticks_A;
volatile uint16_t probe;    //to monitor OCR1A

void setup() 
{
cli();  TCCR1A=0; TCCR1B=0;  
TCCR1B|=(1<<CS12)|(1<<CS10)|(1<<WGM12);//1024 prescale, CTC mode
ticks_A=4000;//store the minimum value of the ticks
OCR1A=ticks_A-1;
TIMSK1|=1<<OCIE1A; //enable timer1 interrupt routine
TCNT1=0;  //initial timer counter
sei();
Serial.begin(9600);
}

void loop() {

Serial.println(probe);
delay(100);

}

ISR(TIMER1_COMPA_vect)//
{ 
 Counter++;
 Counter &=(BITDEP-1); //so Counter is 0,1,2... BITDEP-1

// do something//

 OCR1A<<=1;  //increase OCR1A 

if(0==Counter)
{  
  OCR1A=ticks_A-1; //reset OCR1A to its initial value;
}   
probe=OCR1A;
TCNT1=0; 
}

if BITDEP=4, it works fine and Serial print shows that OCR1A changes as expect. However, if BITDEP=3 or 5, OCR1A=3999 all the times and the code “OCR1A<<=1” doesn’t work. I don’t understand why? Any help or suggestion?

There are errors in your code:

#define BITDEP 5
Volatile uint8_t Counter; //  "Volatile"  Is not a valid type

Change to "volatile"
Problem with BITDEP:

// If BITDEP == 4, BITDEP - 1 is equal to 3 (hex 0x03) and
       Counter &=(BITDEP-1);
// Take the values 0, 1, 2, and 3 only

// But if BITDEP==5, BITDEP-1 is equal to 4 (hex 0x04) and
       Counter &=(BITDEP-1);
// Take the initial value == 0 and success this
 if(0==Counter)
 { 
 OCR1A=ticks_A-1; //reset OCR1A to its initial value;
 }
// The same for BITDEP==3 and odd BITDEP values

This line does what you expect only when BITDEP is a power of 2.

Counter &=(BITDEP-1); //so Counter is 0,1,2... BITDEP-1

You will need to revise the code.

From the comment it appears you intended   Counter &=(BITDEP-1); //so Counter is 0,1,2... BITDEP-1
to do a modulo operation. As mentioned, that will only work for BITDEP = 1,2,4,8,16

You should try using the modulo operator:  Counter %= BITDEP ; //so Counter is 0,1,2... BITDEP-1