How to clean up Serial port buffer

I am sorry, I do not get the purpose of your code - even allowing for it to be a demo/test of a percieved error.

There are two ways of reading Serial data.

The first is to occasionally call Serial.available(), and call Serial.read() to handle the character if it has arrived. If the Serial managed to receive 3 characters between your calling Serial.available() then the 2nd and 3rd byte will still be there on your next check & read call. If you expected 5 bytes, and only 3 arrived .. do something else until calling available again to see if they have arrived.

The other way, is the Serial.event(), which looks to me like an interrupt routine. (Admittedly never used it - never had a need) As it gets called (interrupting what your program was doing) immediatly after a byte is received, there will only be ONE byte to read. Doing Serial.readBytesUntil is going to confuse/break everything.

Unless you are truly confident/knowledgeble with handling interrupts, then stick with the above polling technique.

I have never heard of the AVR serial "getting dirty". And no, you have not discovered a new an exquiste subtle bug.