Arduino -- PC serial connection missing up samples as at high baudrate (python)

Hi, I am fairly new to the world of Arduino. I have setup a system with my main structure as follows :

I have an arduino that is transmitting 10k samples at different baudrates. Arduino is just writing a value at serial port, here is its code :

int i;

void setup() {
  //Initialize serial and wait for port to open:
  //Serial.begin(38400);

  Serial.begin(500000);
  
  while (!Serial) {
    ; // wait for serial port to connect. Needed for native USB port only
  }

  i = 0;
  


}

void loop() {
  
  i++;
     
  Serial.println(i);

  if (i > 1000) {
    
      while(true);  
  }
  
}

and I am reading it on my PC using python, for reading I am using this code :

class ReadLine:
    def __init__(self, s):
        self.buf = bytearray()
        self.s = s

    def readline(self):
        i = self.buf.find(b"\n")
        if i >= 0:
            r = self.buf[:i+1]
            self.buf = self.buf[i+1:]
            return r
        while True:
            i = max(1, min(2048, self.s.in_waiting))
            data = self.s.read(i)
            i = data.find(b"\n")
            if i >= 0:
                r = self.buf + data[:i+1]
                self.buf[0:] = data[i+1:]
                return r
            else:
                self.buf.extend(data)


ser = serial.Serial(
    port='COM4',\
    baudrate=38400,\
        timeout=0)

checker = 0
rl = ReadLine(ser)
while True:
    
   time = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
   print(time)
   print(checker)
   print(rl.readline())
  

   checker += 1
  
ser.close()

Now, if I use the above code and increase the baudrate above 38400, I start missing samples and I am not reading all of the samples on the PC side. Arduino card is Arduino Mega 2560 and I am using a i5 processor in PC. I am using python in Visual Studio.

Can some one please help me out why the samples are getting missed at higher baudrate ?
I have read about the Arduino card and it seems, it can support baudrate upto 500000 easily so it is because PC is lacking behind on serial port ?

Is there some efficient way to read over on the PC side ?
Also, if i use raspberry pi rather than a PC system will it help out for covering the missing samples over serial connection ?

If i use an ethernet connection will it be faster ?

Thank you.

Have a look at this Simple Python - Arduino demo

I regularly communicate between my Arduino and my PC at 500,000 baud with perfect reliability.

For the future, always post complete programs.

...R

Robin2:
Have a look at this Simple Python - Arduino demo

I regularly communicate between my Arduino and my PC at 500,000 baud with perfect reliability.

For the future, always post complete programs.

...R

Thank you, I am looking at the demo. I updated the arduino code to full for python one it is the full code.

Robin2:
Have a look at this Simple Python - Arduino demo

I regularly communicate between my Arduino and my PC at 500,000 baud with perfect reliability.

For the future, always post complete programs.

...R

Just a question out of curiosity, which one is better to test it out with arduino card ? raspberry pi or arduino ?

Also, I am using visual studio for writing the python program, will it be better to use some other ide or write command line python code ?

Sorry i just started in this field so learning on the go. Thank you for your patience.

nothan:
Just a question out of curiosity, which one is better to test it out with arduino card ? raspberry pi or arduino ?

Did you mean "RPi or PC"

I don't see that it would make any difference but I have never had an RPi. However I use Linux on this laptop with all my Arduinos.

...R

Robin2:
Did you mean "RPi or PC"

I don't see that it would make any difference but I have never had an RPi. However I use Linux on this laptop with all my Arduinos.

...R

Thanks, I am using Windows on the system for now so I thought that might be creating some issues. I will try with linux and with RPi.

nothan:
Thanks, I am using Windows on the system for now so I thought that might be creating some issues. I will try with linux and with RPi.

Why? Python is cross-compatible - if it doesn't work on Windows, it surely won't work on Linux.

Power_Broker:
Why? Python is cross-compatible - if it doesn't work on Windows, it surely won't work on Linux.

The problem is not that it is not working the problem is after increasing baudrates above 38400 the samples start dropping off, its like my system is unable to keep up with the baudrate of arduino. While at lesser baudrates it is getting all of the samples, so is it better to switch to SPI communication rather than UART one ?

nothan:
The problem is not that it is not working the problem is after increasing baudrates above 38400 the samples start dropping off,

Have you tried the example in my link in Reply #1 - with a suitable modification of the serial port references the Python code should work on a Windows PC.

...R

nothan:
The problem is not that it is not working the problem is after increasing baudrates above 38400 the samples start dropping off, its like my system is unable to keep up with the baudrate of arduino. While at lesser baudrates it is getting all of the samples, so is it better to switch to SPI communication rather than UART one ?

I find it difficult to believe that your OS is not fast enough to handle higher baud rates unless you're running Windows 98 on a potato...

What is the actual configuration of your PC connection? Is it a true, native, serial port or is it running from a USB port using an adapter cable?

Paul