Arduino Forum

Forum 2005-2010 (read only) => Software => Bugs & Suggestions => Topic started by: eried on Mar 16, 2009, 06:05 am

Title: Road to solve the delay on the Arduino IDE
Post by: eried on Mar 16, 2009, 06:05 am
[size=18]Have been experienced some delays/slowdowns with the arduino ide using Windows? [/size]

Specially when you left your bluetooth serial ports enabled?, when starting or clicking on "TOOLS" menu like you can see here?:

http://screencast.com/t/aBsiowgaA3

After a while playing with the Java code of the IDE, I noticed (http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1236880178) that the problem is a bit more hard to solve than fixing the Arduino IDE, even when I solved another small problem road to my final solution  ::):

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1236999449

You can download the "fixed" Arduino 0014 file here (replace this in the Arduino/lib/ path):
http://servicios.ried.cl/arduino/temp/pde_arduino-0015_fixed_2009-03-15.rar

Well, the problem resides in the rxtx serial library, library that Arduino IDE uses to enumerate the com ports and for communication with the board. This library is open source and multi platform: http://www.rxtx.org/

After hours of trying to find the problem, I isolated the problem: the initialization routines are guilty, because even if I don't call the enumeration (clicking on the TOOLS menu) from the Arduino IDE, the delays appear with any sketch upload or turning on the serial monitor.

The problem begin from this code, in the SerialImp.c file:

Code: [Select]
#ifndef WIN32
     pid = getpid();
#else
     char full_windows_name[80];
#endif /* WIN32 */

     ENTER( "RXTXPort:testRead" );
#ifdef TRENT_IS_HERE_DEBUGGING_ENUMERATION
     /* vmware lies about which ports are there causing irq conflicts */
     /* this is for testing only */
     if( !strcmp( name, "COM1" ) || !strcmp( name, "COM2") )
     {
           printf("%s is good\n",name);
           sprintf( message, "testRead: %s is good!\n", name );
           report( message );
           (*env)->ReleaseStringUTFChars( env, tty_name, name );
           return( JNI_TRUE );
     }
     (*env)->ReleaseStringUTFChars( env, tty_name, name );
     return( JNI_FALSE );
#endif /* TRENT_IS_HERE_DEBUGGING_ENUMERATION */
#ifdef WIN32
     strcpy( full_windows_name, DEVICEDIR );
     strcat( full_windows_name, name );
     ret = serial_test((char *) full_windows_name );
     
     ret = serial_test((char *) name );
     (*env)->ReleaseStringUTFChars( env, tty_name, name );
     return(ret);
#endif /* WIN32 *


For each com port, we call "serial_test((char *) full_windows_name );" to check if we can open that port. In Windows each com port, device, etc is a file too (like in Unix, but Windows does not allow us to see this directly using a "class-device" approach, not a file-based one), ports as any other device are part of the "//./" namespace, for example "//./COM12" is the path for my LOCALHOST "." COM12 port. If the system can open this port/file, I will treat this port as valid, in the termios.c file:

Code: [Select]
         unsigned long *hcomm;
         hcomm = CreateFile( filename, GENERIC_READ |GENERIC_WRITE, 0, 0, OPEN_EXISTING, 0, 0 );
         if ( hcomm == INVALID_HANDLE_VALUE )
         {
               if (GetLastError() == ERROR_ACCESS_DENIED)
               {
                     ret = 1;
               }
               else
               {
                     ret = 0;
               }
         }
         else
         {
               ret = 1;
         }
         CloseHandle( hcomm );


Opening a file... yeah right but there is a small issue with this method. We want to check a lot of port quicky (each time that the Arduino/Tool menu is shown, uploaded a sketch, etc) so we need a non blocking operation, something like Synchronous vs Asynchronous (http://msdn.microsoft.com/en-us/library/aa365683(VS.85).aspx), but for this task CreateFile is very buggy. It delays if the "file" does not respond until a timeout, so if my bluetooth dongle is connected and there are some virtual bt com ports, the process is very laggy and time-consuming.

Thanks to http://www.codeproject.com/KB/system/enumports.aspx I discovered another way to check ports. It is only for Windows NT, so we need to keep the compatibility backwards. Even when the new function is a lot faster, its very dirty (not as the old solution) because to keep the compatibility I just overwrite the "serial_test" function, and this function only works for one port at once. But in fact, using the QueryDosDevice API (for NT) I have the full list of devices at once, not one by one (so in the future the next release of this rxtx library will be a lot quicker).

Here is the brand-new ultra fast serial_test ;):

Code: [Select]
/*----------------------------------------------------------
serial_test

  accept: filename to test
  perform:
  return:      1 on success 0 on failure
  exceptions:
  win32api:    CreateFile CloseHandle
  comments:    if the file opens it should be ok.
----------------------------------------------------------*/
int serial_test( char * filename )
{
   int ret = 0;
   
   // Getting the Windows Version
   OSVERSIONINFO osvi;
   osvi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
   BOOL bGetVer = GetVersionEx(&osvi);
   
   // Using the QueryDosDevice API (on NT)
   if (bGetVer && (osvi.dwPlatformId == VER_PLATFORM_WIN32_NT))
   {
       // This solution is based on http://www.codeproject.com/KB/system/enumports.aspx
       TCHAR szDevices[65535];
       DWORD dwChars = QueryDosDevice(NULL, szDevices, 65535);
       
       if (dwChars)
       {
         int i=0;
             
         for (;;)
         {
           //Get the current device name
           char* pszCurrentDevice = &szDevices[i];

           if (strlen(pszCurrentDevice) > 3 && strcmp(pszCurrentDevice,filename)==0)
           {
               ret = 1;
               break;
           }
           
           // Go to next NULL character
           while(szDevices[i] != '\0')
             i++;
           
           // Bump pointer to the next string
           i++;
           
           // The list is double-NULL terminated, so if the character is
           // now NULL, we're at the end
           if (szDevices[i] == '\0')
             break;
         }
       }
   }
   else
   {
       // Buggy way to test if we can open the comport (on Win9x)
         unsigned long *hcomm;
         hcomm = CreateFile( filename, GENERIC_READ |GENERIC_WRITE, 0, 0, OPEN_EXISTING, 0, 0 );
         if ( hcomm == INVALID_HANDLE_VALUE )
         {
               if (GetLastError() == ERROR_ACCESS_DENIED)
               {
                     ret = 1;
               }
               else
               {
                     ret = 0;
               }
         }
         else
         {
               ret = 1;
         }
         CloseHandle( hcomm );
   }
     return(ret);
}


And the new and fixed compiled library is here (for all those have problems with delays and slowdowns in their arduino IDE, startup, uploading or displaying the TOOLS menu, any build, just replace the file "rxtxSerial.dll")
http://servicios.ried.cl/arduino/temp/rxtxSerial-2.2_fixed_2009-03-17.rar

Hope it helps for your problem/ or as experience to solve other problems. The key is a whole night, Pepsi and reading the README files!
Title: Re: Road to solve the delay on the Arduino IDE
Post by: dcb on Mar 16, 2009, 06:31 am
That sucks, nobody wants to have to maintain windows specific code I'm sure.   How long is the delay anyway?  (video didn't work under firefox/ubuntu).  

So even knowing which port you want won't help?
Title: Re: Road to solve the delay on the Arduino IDE
Post by: eried on Mar 16, 2009, 06:38 am
This is the problem with this psedo-multiplatform solutions.

The delay on my machine was about 30 seconds at startup, then 20-30 seconds everytime I open the TOOLS menu to choose a port.

Now, with the fixed library, its barely noticeable (0.1 sec or less).

I was using the windows console to upload the sketches because this delay (I often use Bluetooth so disabling it every time I use arduino was a worse solution than using the console), even I am thinking on code a C# new arduino IDE, but until this moment I haven't found a free richtextbox component with visual studio-kinda capabilities, collapsible sourcecode, intellisense, etc.

I'm very happy with the results anyway, even when the creator of that library was aware of this error. (now a lot of projects using rxtx will be updated with the version 2.2 of this library thanks to this small research)

Knowing the port would help (this way you did not need to scan those bt-related laggy ones), but the way arduino IDE is, its necessary to scan all ports to show the TOOLS/SERIAL PORT submenu
Title: Re: Road to solve the delay on the Arduino IDE
Post by: dcb on Mar 16, 2009, 06:08 pm
"This is the problem with this psedo-multiplatform solutions."

You can safely blame windows for that, as evidenced here you have to do it their way or it sucks.  Sometimes I think they pay people to break stuff.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: m_elias on Mar 16, 2009, 11:32 pm
I tried your fixed library this morning and it worked perfectly. Later during the day, I was programming a Sanguino with my tablet & the new library and it seemed to still have some delays. The delays were not as long as before, but also not as short as when I tested it in the morning. Just now I tested the new library again and, once again it was working quite well.

Much appreciated!
Title: Re: Road to solve the delay on the Arduino IDE
Post by: eried on Mar 17, 2009, 01:09 am
I will try to release a even better fixed library to reduce those small lags
Title: Re: Road to solve the delay on the Arduino IDE
Post by: eried on Mar 17, 2009, 06:41 am
New update:

http://servicios.ried.cl/arduino/temp/rxtxSerial-2.2_fixed_2009-03-17.rar

(only for WinNT, 2000, XP, Vista or newer only)
Title: Re: Road to solve the delay on the Arduino IDE
Post by: pghislain on Jun 09, 2009, 09:43 pm
it works 100% thanks 4 the Master GuRu from Chile! Viva Eried!
Title: Re: Road to solve the delay on the Arduino IDE
Post by: Siegfried Loeffler on Sep 11, 2009, 06:20 pm
I have that same problem with the arduino-0017 IDE on Windows. Incredible delays every time I open a serial monitor, upload something, or when I startup the application.

Has this fixed library been included in the 0017 IDE ? Will it work with 0017?

Thanks
Title: Re: Road to solve the delay on the Arduino IDE
Post by: agib on Sep 12, 2009, 12:18 am
Just replying to say THANKS!

Your fix worked for me using 0017 and Windows 7 (RC).
Title: Re: Road to solve the delay on the Arduino IDE
Post by: macegr on Nov 24, 2009, 11:58 pm
Devs: please include this update in Arduino 0018. The enhanced rxtx solves one of the most annoying problems I've run into in the Arduino IDE. If you have any sleeping Bluetooth SPP devices, the default Arduino IDE will take a long time to load, and then a long time to open the Tools menu every time.

In addition, your sleeping Bluetooth SPP connections are trashed. The old rxtx somehow sits on them and prevents them from being used by other applications. Just by opening the Arduino IDE once, you now have to reset multiple Bluetooth devices, or unpair/repair them.

The new rxtx dll above fixes that issue completely.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: Osgeld on Nov 25, 2009, 01:01 am
is this a all windows thing or exclusive to vista + (wondering cause i have it on both xp 32 bit and xp 64 bit and its been working fine)
Title: Re: Road to solve the delay on the Arduino IDE
Post by: macegr on Nov 25, 2009, 04:35 am
If you don't have bluetooth serial devices then you probably aren't going to notice anything.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: FrankM on Nov 27, 2009, 11:11 am
Yes, yes, yes, please add this to 0018! The difference is dramatic! My startup goes from a little below 1 minute to only several seconds. Opening the tools menu even took me two minutes (no joke) and now it shows immediately.

(Thanks to the OP) [smiley=thumbsup.gif] [smiley=thumbsup.gif] [smiley=thumbsup.gif]
Title: Re: Road to solve the delay on the Arduino IDE
Post by: saerdna on Jan 16, 2010, 06:05 pm
Hello togheter,

I have also problems with my arduino17 on a vista pc. I would try this fix, but cannot open the rar file.
Can anybody give me a hint, how to open this file,
Thank's

andi
Title: Re: Road to solve the delay on the Arduino IDE
Post by: TBAr on Jan 16, 2010, 10:17 pm
Quote
I would try this fix, but cannot open the rar file.
Can anybody give me a hint, how to open this file

http://www.rarlab.com/

Also, from http://en.wikipedia.org/wiki/Rar#Software:

Quote
Third party programs that can only read (unpack) RAR files include: WinZip (Windows), 7-Zip (multiplatform), IZArc (Windows), PeaZip (Windows, Linux), Zipeg (Windows, Mac OS X), ALZip (Windows), along with the free version of unrar (which is available for Linux and FreeBSD). Mac OS X readers include Stuffit Expander, The Unarchiver and Zipeg. Stuffit Expander is also available for Mac OS Classic with RAR support for this platform.


If you go to the second link above, all those third party programs have their own links. I'm too lazy on a Saturday to copy the links into this reply.  ;)
Title: Re: Road to solve the delay on the Arduino IDE
Post by: eried on Jan 30, 2010, 10:11 am
My fixed dll is still working with the version 0018 :) and of course for those who asked, with the 0017 too.  ;D
Title: Re: Road to solve the delay on the Arduino IDE
Post by: ArdJim on Jun 14, 2010, 10:17 pm
Another big thanks for fixing this problem.  I have an older laptop running XP, and the arduino application was completely unusable due to the delays in the 0018 distribution DLL.

Your modified DLL fixed the entire problem, instantly.  8-)

Jim
Title: Re: Road to solve the delay on the Arduino IDE
Post by: crodgers on Aug 31, 2010, 04:56 pm
Dear eried,

Please could I have the source code to your modified version of the rxtx library?

Thanks,

Chris.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: eried on Sep 01, 2010, 08:55 am
I will try to look for it. But you can build your own patched version. First look the point 7 of this document, ignore the text, just copy the "serial_test" function code.

Then get the full source from http://users.frii.com/jarvi/rxtx/

And finally, replace the code and Compile.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: johngineer on Sep 05, 2010, 02:28 am
Works for me using Arduino 0019!

Thank you so much, eried! You've done us all a great favor! Nice work!
Title: Re: Road to solve the delay on the Arduino IDE
Post by: chrlilje on Oct 26, 2010, 12:14 am
Yet another very happy user, thanks a lot!

The fix - It works with Processing too, btw.

I am trying to get a link to this fix included in the
http://www.arduino.cc/en/Guide/Troubleshooting#toc14

.. I hope this will be possible. I remembered seeing this solution long time ago, but I had trouble refinding it.

More links to this genious :-)

with humble respect
Christian Liljedahl
Title: Re: Road to solve the delay on the Arduino IDE
Post by: ocon on Nov 03, 2010, 02:20 am
WOW...
I have same problem.
This is very fantastic solution.

Many thanks!!!
Title: Re: Road to solve the delay on the Arduino IDE
Post by: skal on Nov 05, 2010, 08:08 pm
Hey all, i just got my UNO yesterday and noticed the delay on the IDE soft and i can tell that this fix worked for the 0021, too(i had BT com port working on my laptop). thanks!  :)
Title: Re: Road to solve the delay on the Arduino IDE
Post by: perasperaadastra on Nov 29, 2010, 03:32 am
This helped me a lot!!! Thank you very much!!!

//Arduino UNO - 021 - Win7
Title: Re: Road to solve the delay on the Arduino IDE
Post by: alearduin on Dec 28, 2010, 10:15 pm
gracias por el archivo! ha mejorado la respuesta del programa muy mucho!

thanks for the great job done! I had the same problem in Processing (as they share the same platform) and also solved it.
Title: Re: Road to solve the delay on the Arduino IDE
Post by: and on Jan 23, 2011, 01:05 am
:)  Luckily I found this thread.

Eried, thank you for this fix; extremely helpful. Turned an almost unusable program into a valuable tool.

works as well with Arduino 022.

Thanks,

Andreas
Title: Re: Road to solve the delay on the Arduino IDE
Post by: flav on Oct 23, 2014, 02:35 pm
Hello Mr ERied.

I have the same problem with the slow Arduino IDE.  I'm new at this and don't yet know Java.  When I look at the Arduino\Lib directory there are only *.jar files.  I do not know what to do with your *.rar file.  Could you please give more detailed instructions for us Java newbies.  Gracias in advance.
Flav
Title: Re: Road to solve the delay on the Arduino IDE
Post by: flav on Oct 23, 2014, 03:56 pm
My bad.  I downloaded 7-zip to unzip your *rar file and replaced the rxtxSerial.dll iin the Arduino directory.
However, there seems to be no change in the excessive compile time using the arduino IDE.  I am using windows 8.1.  Any thoughts and suggestions?
Flav
Title: Re: Road to solve the delay on the Arduino IDE
Post by: flav on Oct 23, 2014, 09:48 pm
In addition to using Mr. Ried's solution, I had to turn off Spybot's live protection.  Now all is well!! 
Thank you again Mr. E. Ried!!