Go Down

Topic: Road to solve the delay on the Arduino IDE (Read 48704 times) previous topic - next topic

eried

Mar 16, 2009, 06:05 am Last Edit: Mar 17, 2009, 06:37 am by eried Reason: 1
[size=18]Have been experienced some delays/slowdowns with the arduino ide using Windows? [/size]

Specially when you left your bluetooth serial ports enabled?, when starting or clicking on "TOOLS" menu like you can see here?:

http://screencast.com/t/aBsiowgaA3

After a while playing with the Java code of the IDE, I noticed (http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1236880178) that the problem is a bit more hard to solve than fixing the Arduino IDE, even when I solved another small problem road to my final solution  ::):

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1236999449

You can download the "fixed" Arduino 0014 file here (replace this in the Arduino/lib/ path):
http://servicios.ried.cl/arduino/temp/pde_arduino-0015_fixed_2009-03-15.rar

Well, the problem resides in the rxtx serial library, library that Arduino IDE uses to enumerate the com ports and for communication with the board. This library is open source and multi platform: http://www.rxtx.org/

After hours of trying to find the problem, I isolated the problem: the initialization routines are guilty, because even if I don't call the enumeration (clicking on the TOOLS menu) from the Arduino IDE, the delays appear with any sketch upload or turning on the serial monitor.

The problem begin from this code, in the SerialImp.c file:

Code: [Select]
#ifndef WIN32
     pid = getpid();
#else
     char full_windows_name[80];
#endif /* WIN32 */

     ENTER( "RXTXPort:testRead" );
#ifdef TRENT_IS_HERE_DEBUGGING_ENUMERATION
     /* vmware lies about which ports are there causing irq conflicts */
     /* this is for testing only */
     if( !strcmp( name, "COM1" ) || !strcmp( name, "COM2") )
     {
           printf("%s is good\n",name);
           sprintf( message, "testRead: %s is good!\n", name );
           report( message );
           (*env)->ReleaseStringUTFChars( env, tty_name, name );
           return( JNI_TRUE );
     }
     (*env)->ReleaseStringUTFChars( env, tty_name, name );
     return( JNI_FALSE );
#endif /* TRENT_IS_HERE_DEBUGGING_ENUMERATION */
#ifdef WIN32
     strcpy( full_windows_name, DEVICEDIR );
     strcat( full_windows_name, name );
     ret = serial_test((char *) full_windows_name );
     
     ret = serial_test((char *) name );
     (*env)->ReleaseStringUTFChars( env, tty_name, name );
     return(ret);
#endif /* WIN32 *


For each com port, we call "serial_test((char *) full_windows_name );" to check if we can open that port. In Windows each com port, device, etc is a file too (like in Unix, but Windows does not allow us to see this directly using a "class-device" approach, not a file-based one), ports as any other device are part of the "//./" namespace, for example "//./COM12" is the path for my LOCALHOST "." COM12 port. If the system can open this port/file, I will treat this port as valid, in the termios.c file:

Code: [Select]
         unsigned long *hcomm;
         hcomm = CreateFile( filename, GENERIC_READ |GENERIC_WRITE, 0, 0, OPEN_EXISTING, 0, 0 );
         if ( hcomm == INVALID_HANDLE_VALUE )
         {
               if (GetLastError() == ERROR_ACCESS_DENIED)
               {
                     ret = 1;
               }
               else
               {
                     ret = 0;
               }
         }
         else
         {
               ret = 1;
         }
         CloseHandle( hcomm );


Opening a file... yeah right but there is a small issue with this method. We want to check a lot of port quicky (each time that the Arduino/Tool menu is shown, uploaded a sketch, etc) so we need a non blocking operation, something like Synchronous vs Asynchronous (http://msdn.microsoft.com/en-us/library/aa365683(VS.85).aspx), but for this task CreateFile is very buggy. It delays if the "file" does not respond until a timeout, so if my bluetooth dongle is connected and there are some virtual bt com ports, the process is very laggy and time-consuming.

Thanks to http://www.codeproject.com/KB/system/enumports.aspx I discovered another way to check ports. It is only for Windows NT, so we need to keep the compatibility backwards. Even when the new function is a lot faster, its very dirty (not as the old solution) because to keep the compatibility I just overwrite the "serial_test" function, and this function only works for one port at once. But in fact, using the QueryDosDevice API (for NT) I have the full list of devices at once, not one by one (so in the future the next release of this rxtx library will be a lot quicker).

Here is the brand-new ultra fast serial_test ;):

Code: [Select]
/*----------------------------------------------------------
serial_test

  accept: filename to test
  perform:
  return:      1 on success 0 on failure
  exceptions:
  win32api:    CreateFile CloseHandle
  comments:    if the file opens it should be ok.
----------------------------------------------------------*/
int serial_test( char * filename )
{
   int ret = 0;
   
   // Getting the Windows Version
   OSVERSIONINFO osvi;
   osvi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
   BOOL bGetVer = GetVersionEx(&osvi);
   
   // Using the QueryDosDevice API (on NT)
   if (bGetVer && (osvi.dwPlatformId == VER_PLATFORM_WIN32_NT))
   {
       // This solution is based on http://www.codeproject.com/KB/system/enumports.aspx
       TCHAR szDevices[65535];
       DWORD dwChars = QueryDosDevice(NULL, szDevices, 65535);
       
       if (dwChars)
       {
         int i=0;
             
         for (;;)
         {
           //Get the current device name
           char* pszCurrentDevice = &szDevices[i];

           if (strlen(pszCurrentDevice) > 3 && strcmp(pszCurrentDevice,filename)==0)
           {
               ret = 1;
               break;
           }
           
           // Go to next NULL character
           while(szDevices[i] != '\0')
             i++;
           
           // Bump pointer to the next string
           i++;
           
           // The list is double-NULL terminated, so if the character is
           // now NULL, we're at the end
           if (szDevices[i] == '\0')
             break;
         }
       }
   }
   else
   {
       // Buggy way to test if we can open the comport (on Win9x)
         unsigned long *hcomm;
         hcomm = CreateFile( filename, GENERIC_READ |GENERIC_WRITE, 0, 0, OPEN_EXISTING, 0, 0 );
         if ( hcomm == INVALID_HANDLE_VALUE )
         {
               if (GetLastError() == ERROR_ACCESS_DENIED)
               {
                     ret = 1;
               }
               else
               {
                     ret = 0;
               }
         }
         else
         {
               ret = 1;
         }
         CloseHandle( hcomm );
   }
     return(ret);
}


And the new and fixed compiled library is here (for all those have problems with delays and slowdowns in their arduino IDE, startup, uploading or displaying the TOOLS menu, any build, just replace the file "rxtxSerial.dll")
http://servicios.ried.cl/arduino/temp/rxtxSerial-2.2_fixed_2009-03-17.rar

Hope it helps for your problem/ or as experience to solve other problems. The key is a whole night, Pepsi and reading the README files!
My website: http://ried.cl

dcb

That sucks, nobody wants to have to maintain windows specific code I'm sure.   How long is the delay anyway?  (video didn't work under firefox/ubuntu).  

So even knowing which port you want won't help?

eried

#2
Mar 16, 2009, 06:38 am Last Edit: Mar 16, 2009, 06:44 am by eried Reason: 1
This is the problem with this psedo-multiplatform solutions.

The delay on my machine was about 30 seconds at startup, then 20-30 seconds everytime I open the TOOLS menu to choose a port.

Now, with the fixed library, its barely noticeable (0.1 sec or less).

I was using the windows console to upload the sketches because this delay (I often use Bluetooth so disabling it every time I use arduino was a worse solution than using the console), even I am thinking on code a C# new arduino IDE, but until this moment I haven't found a free richtextbox component with visual studio-kinda capabilities, collapsible sourcecode, intellisense, etc.

I'm very happy with the results anyway, even when the creator of that library was aware of this error. (now a lot of projects using rxtx will be updated with the version 2.2 of this library thanks to this small research)

Knowing the port would help (this way you did not need to scan those bt-related laggy ones), but the way arduino IDE is, its necessary to scan all ports to show the TOOLS/SERIAL PORT submenu
My website: http://ried.cl

dcb

#3
Mar 16, 2009, 06:08 pm Last Edit: Mar 16, 2009, 06:09 pm by dcb Reason: 1
"This is the problem with this psedo-multiplatform solutions."

You can safely blame windows for that, as evidenced here you have to do it their way or it sucks.  Sometimes I think they pay people to break stuff.

Matt Elias

I tried your fixed library this morning and it worked perfectly. Later during the day, I was programming a Sanguino with my tablet & the new library and it seemed to still have some delays. The delays were not as long as before, but also not as short as when I tested it in the morning. Just now I tested the new library again and, once again it was working quite well.

Much appreciated!

eried

I will try to release a even better fixed library to reduce those small lags
My website: http://ried.cl

eried

#6
Mar 17, 2009, 06:41 am Last Edit: Mar 17, 2009, 06:42 am by eried Reason: 1
New update:

http://servicios.ried.cl/arduino/temp/rxtxSerial-2.2_fixed_2009-03-17.rar

(only for WinNT, 2000, XP, Vista or newer only)
My website: http://ried.cl

pghislain

#7
Jun 09, 2009, 09:43 pm Last Edit: Jun 09, 2009, 09:45 pm by pghislain Reason: 1
it works 100% thanks 4 the Master GuRu from Chile! Viva Eried!
Pierre Ghislain
TopWare
http://www.topware.pt

Siegfried Loeffler

I have that same problem with the arduino-0017 IDE on Windows. Incredible delays every time I open a serial monitor, upload something, or when I startup the application.

Has this fixed library been included in the 0017 IDE ? Will it work with 0017?

Thanks

agib

Just replying to say THANKS!

Your fix worked for me using 0017 and Windows 7 (RC).

macegr

Devs: please include this update in Arduino 0018. The enhanced rxtx solves one of the most annoying problems I've run into in the Arduino IDE. If you have any sleeping Bluetooth SPP devices, the default Arduino IDE will take a long time to load, and then a long time to open the Tools menu every time.

In addition, your sleeping Bluetooth SPP connections are trashed. The old rxtx somehow sits on them and prevents them from being used by other applications. Just by opening the Arduino IDE once, you now have to reset multiple Bluetooth devices, or unpair/repair them.

The new rxtx dll above fixes that issue completely.
Unique RGB LED Modules and Arduino shields: http://www.macetech.com/store

Osgeld

is this a all windows thing or exclusive to vista + (wondering cause i have it on both xp 32 bit and xp 64 bit and its been working fine)
http://arduino.cc/forum/index.php?action=unread;boards=2,3,4,5,67,6,7,8,9,10,11,66,12,13,15,14,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,86,87,89,1;ALL

macegr

If you don't have bluetooth serial devices then you probably aren't going to notice anything.
Unique RGB LED Modules and Arduino shields: http://www.macetech.com/store

FrankM

Yes, yes, yes, please add this to 0018! The difference is dramatic! My startup goes from a little below 1 minute to only several seconds. Opening the tools menu even took me two minutes (no joke) and now it shows immediately.

(Thanks to the OP) [smiley=thumbsup.gif] [smiley=thumbsup.gif] [smiley=thumbsup.gif]

andi

Hello togheter,

I have also problems with my arduino17 on a vista pc. I would try this fix, but cannot open the rar file.
Can anybody give me a hint, how to open this file,
Thank's

andi

Go Up