That's because the while loop does nothing but count until the timer expires and the if runs the entire loop() function until the timer expires. The if is slowed down by the time it takes to exit and re-enter the loop() function.
Note that you are also measuring the time it takes to call millis() and calculate if the time has expired.
Rather than running the loop for one second you could run the loop for a million counts and compare the start and end times. This would be two calls to millis() instead of 130,000 calls.
unsigned long startTime, endTime;
unsigned long counter = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
startTime = millis();
while (counter < 1000000) counter++;
endTime = millis();
Serial.print("Speed of calculation: ");
Serial.print(1000000000UL / (endTime - startTime));
Serial.println(" Hz");
}