ESP32 / Big Sur Mac OS compilation fail

I’ve been researching this bug, which apparently is an issue in Pyserial querying available ports. The errors seen when compiling for ESP32 is typically this:

Traceback (most recent call last):
  File "esptool.py", line 57, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports.py", line 29, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports_posix.py", line 31, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "serial/tools/list_ports_osx.py", line 32, in <module>
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found
Failed to execute script esptool
exit status 255
Error compiling for board ESP32 Dev Module.

I’ve tried various fixes which include editing ESPTOOL.PY, but all to no avail, and to be honest can’t make much sense of the reports on GitHub which say that the issues with Pyserial are closed.

Does anyone have a step-by-step guide of the fix, please? I’ve seen this, but it doesn’t really appear to be very complete.

1. Comment below lines in the file "/Library/Frameworks/Python.framework/Versions/3.8/bin/esptool.py"
#try:
#    import serial.tools.list_ports as list_ports
#except ImportError:
#    print("The installed version (%s) of pyserial appears to be too old for esptool.py (Python interpreter %s). "
#          "Check the README for installation instructions." % (sys.VERSION, sys.executable))
#    raise
2. List all available ports
ls /dev/tty.*
3. Use esptool.py with --port command
esptool.py --port /dev/tty.usbserial-01EED907 --baud 921600 erase_flash
4. Update esptool.py or env variable with a default port so esptool.py can be called without a parameter.
TODO

I’m using a PC at the moment for my project, but it’s slower and clumsier… Obviously, this needs to be fixed in time.

I should point out that ATMega Arduino projects compile without issues, so it is a problem in the ESP32 tool chain.

I just bought some ESP32 boards, running into the exact same problem and just wondering if you ever found a fix?

Yes, I followed the tips here: MacOS Big Sur dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found (ESPTOOL-82) · Issue #540 · espressif/esptool · GitHub

Pyserial is the culprit.

So you were able to actually get it working or we have to just wait for pyserial update?

Yes, it’s working.

BigSur broke a LOT of things

Arduino engineers became aware of the larger picture this morning but don't expect any quick fix for SNAPples mistake.

If bigsur can be rolled back you may want to try that ?

Also the SEARCH for "big sur" on this forum may bring you a couple of work arounds worth playing with.

It looks like the esptool is a compiled version, not the Python one. I did the following:

copy /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/tools/esptool.py
to /Users/XXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py

chmod +x esptool.py

edit the file /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/platform.txt
and change the following line:
tools.esptool_py.cmd=esptool into:
tools.esptool_py.cmd=esptool.py

save platform.txt, restart ArduinoIDE

That did not work for me actually. I’ve made the changes as in previous post, but only to get:

Traceback (most recent call last):
  File "/Users/XXXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 37, in <module>
    import serial
ImportError: No module named serial
exit status 1
Error compiling for board AI Thinker ESP32-CAM.

What I needed to do aditionally, is to force the use of Python 3 on my machine, but updating in esptool.py the line:

#!/usr/bin/env python

into

#!/usr/bin/env python3

@bjhemstra solution works for me, but make sure you are a superuser before you use the “chmod +x esptool.py”. You can become superuser bij typing this command “sudo su”. Also make sure you are in the right file directory for doing this command.

Thnx, @bjhemstra for your help

Good morning to everybody,

this post is just to thank @bjhemstra for the effective solution he gave to this annoying problem. I resisted slightly more than a month before upgrading to Big Sur then, being affected by the MUS (Maniac Upgrade Syndrome ), I did it and fell in troubles immediately after.
I have applied @bjhemstra's solution to both my iMac 27" (late 2015) as well as to my wife's MacBook Pro (2017) ,which I keep as a backup.
Everything went smoothly, I didn't even have to force the use of Python 3.

Many thanks @bjhemstra.

Best wishes for 2021

After applying @bjhemstra’s fix (thanks for that!), I also got:

Traceback (most recent call last):
  File "/Users/XXXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py", line 37, in <module>
    import serial
ImportError: No module named serial
exit status 1
Error compiling for board AI Thinker ESP32-CAM.

I didn’t have the python serial library for either version 2 or 3 so the ‘python3’ fix didn’t work, but the following did:

sudo easy_install pyserial

All is now lovely.

hi
before the Update to MAC OS Big Sur, I could compile my sketches, but not more now!
I have read a lot about this issue:

Traceback (most recent call last):
File “esptool.py”, line 57, in
File “/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py”, line 389, in load_module
File “serial/tools/list_ports.py”, line 29, in
File “/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py”, line 389, in load_module
File “serial/tools/list_ports_posix.py”, line 31, in
File “/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py”, line 389, in load_module
File “serial/tools/list_ports_osx.py”, line 32, in
ValueError: dlsym(RTLD_DEFAULT, kIOMasterPortDefault): symbol not found
Failed to execute script esptool
Using library WiFi at version 1.2.7 in folder: /Applications/Arduino-2.app/Contents/Java/libraries/WiFi
Using library SPI at version 1.0 in folder: /Users/maxmueller/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/SPI
exit status 255
/Applications/Arduino-2.app/Contents/Java/arduino-builder returned 255
Error compiling for board AI Thinker ESP32-CAM.

I have tried a lot of all this suggestions, but I can’t resolve the issue, why???
incl. >>> sudo easy_install pyserial
I would very glad, if I could get a guidance to solve
many thank in advance
Max

Desse jeito funciono pra mim

rogerjane:
After applying @bjhemstra’s fix (thanks for that!), I also got:

Traceback (most recent call last):

File “/Users/XXXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py”, line 37, in
   import serial
ImportError: No module named serial
exit status 1
Error compiling for board AI Thinker ESP32-CAM.




I didn't have the python serial library for either version 2 or 3 so the 'python3' fix didn't work, but the following did:



sudo easy_install pyserial




All is now lovely.

After playing around with @bjhemstra’s fix for awhile and restarting Arduino IDE a lot, I finally got past the OP’s problem. I had to download esptool.py from Github and then copied esptool.py from downloads into my /Users/XXXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/ folder. Next I vi’d esptool.py, changing python to python3 as shown in #7. Then I had to install Xcode, and then I hit this serial error. Oddly, if I run pip freeze, it says

...
esptool==3.0
...
pyserial==3.5
...

serial==0.0.97

Which seems to say serial is installed, but the code still fails to compile.

bjhemstra:
It looks like the esptool is a compiled version, not the Python one. I did the following:

copy /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/tools/esptool.py
to /Users/XXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py

chmod +x esptool.py

edit the file /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/platform.txt
and change the following line:
tools.esptool_py.cmd=esptool into:
tools.esptool_py.cmd=esptool.py

save platform.txt, restart ArduinoIDE

bjhemstra:
It looks like the esptool is a compiled version, not the Python one. I did the following:

copy /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/tools/esptool.py
to /Users/XXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py

chmod +x esptool.py

edit the file /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/platform.txt
and change the following line:
tools.esptool_py.cmd=esptool into:
tools.esptool_py.cmd=esptool.py

save platform.txt, restart ArduinoIDE

:)You saved my spirit and my xss :slight_smile:

I am also running into these problems. I've made the changes up to #13 but am still getting

ModuleNotFoundError: No module named 'serial'
exit status 1
Error compiling for board NodeMCU-32S.

Has anyone been able to find a fix for this? This is the first time I've been using Arduino IDE and ESP32 so very inexperienced.

wow, what experts! I had the same problem but upgraded to "big slur" before I received my ESP32 boards. Thank god for this forum!
Special thanks to bjhemstra for step #6 and rogerjane for step #10, both of which were necessary to get me running.
great work folks and thank you.

The solution by juanssl in this thread fixed my issue perfectly. check it out

bjhemstra:
It looks like the esptool is a compiled version, not the Python one. I did the following:

copy /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/tools/esptool.py
to /Users/XXXXX/Library/Arduino15/packages/esp32/tools/esptool_py/2.6.1/esptool.py

chmod +x esptool.py

edit the file /Users/XXXXX/Library/Arduino15/packages/esp32/hardware/esp32/1.0.4/platform.txt
and change the following line:
tools.esptool_py.cmd=esptool into:
tools.esptool_py.cmd=esptool.py

save platform.txt, restart ArduinoIDE

I can confirm this solution is working! Thanks!

I signed up to this forum (after some lurking) just so I could thank bjhemstra for step #6 and rogerjane for step #10, too!
Thank you!

Now I can play...