Ok I figured it out, the RFID reader is continuing so send serial data no matter how long I wait from a single card read. I added a little loop to wait till the reader stops sending data before continuing on with my program like this.
while (rfidSerial.available() > 0){
//Sit and do nothing and wait for reader to be empyty
rfidSerial.read();
Serial.println(i);
i++;
delay(100);
}