Python crashing on MacOS 10.15 Beta (19A582a) with "/usr/lib/libcrypto.dylib"

I ran my Django project with new macOS Catalina and was running fine.
I installed oh_my_zsh then I tried to run the same project it is crashing with the following errors. I uninstalled oh_my_zsh and tried again but it did not worked.

Path:                  /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.4 (3.7.4)
Code Type:             X86-64 (Native)
Parent Process:        Python [7526]
Responsible:           Terminal [7510]
User ID:               501

Date/Time:             2019-10-07 20:59:20.675 +0530
OS Version:            Mac OS X 10.15 (19A582a)
Report Version:        12
Anonymous UUID:        CB7F20F6-96C0-4F63-9EC5-AFF3E0989687


Time Awake Since Boot: 3000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

I just came across the same problem and felt a bit uncomfortable to manually link things around.

I was able to solve the problem by simply

1. Installing openssl via homebrew:

   brew install openssl
   

2. Pointing towards the dynamic libraries from openssl via DYLD_LIBRARY_PATH:

   export DYLD_LIBRARY_PATH=/usr/local/opt/openssl/lib:$DYLD_LIBRARY_PATH
   

I've just added that line to my .zshrc.

Edit: According to this question, the usage of DYLD_FALLBACK_LIBRARY_PATH might be preferable over DYLD_LIBRARY_PATH.

Edit 2: As mentioned in a comment below, this should probably be the accepted answer. Simply reinstall the cryptography package.

For me it was enough to re-install Python's cryptography package.

pip uninstall cryptography
pip install cryptography

Caveat: I am not a security expert, and this solution messes with crypto libraries!

I don't think your issue stems from zsh or oh-my-zsh. My best guess: some crypto libraries installed with MacOS 10.15 are incompatible with Homebrew's python3 installation.

Here's what fixed the issue for me

# Install openssl via homebrew.
# Note: According to homebrew, "openssl is keg-only, which means it was
# not symlinked into /usr/local, because Apple has deprecated use of
# OpenSSL in favor of its own TLS and crypto libraries."
brew install openssl
# Symlink those versions into /usr/local/lib, which gets Python to dynamically
# link against those instead of the version in /usr/lib/.
# Got the idea from https://forums.developer.apple.com/thread/119429
cd /usr/local/lib
sudo ln -s /usr/local/Cellar/openssl/1.0.2t/lib/libssl.1.0.0.dylib libssl.dylib
sudo ln -s /usr/local/Cellar/openssl/1.0.2t/lib/libcrypto.1.0.0.dylib libcrypto.dylib

My situation for context:

• Recently upgraded to MacOS 10.15
• I use python/pip installed via homebrew: brew install python
• pip3 was failing with SIGABRT

Header of system error report:

Process:               Python [52429]
Path:                  /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.4 (3.7.4)
Code Type:             X86-64 (Native)
Parent Process:        zsh [43309]
Responsible:           iTerm2 [2316]
User ID:               501

Date/Time:             2019-10-09 09:52:18.148 -0700
OS Version:            Mac OS X 10.15 (19A583)
Report Version:        12
Bridge OS Version:     4.0 (17P572)
Anonymous UUID:        

Sleep/Wake UUID:       

Time Awake Since Boot: 9900 seconds
Time Since Wake:       7300 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

I prefer a combination of @bixel, @Juro Oravec & @honkaboy answers:

brew install openssl
cd /usr/local/lib
sudo ln -s /usr/local/opt/openssl/lib/libssl.dylib libssl.dylib
sudo ln -s /usr/local/opt/openssl/lib/libcrypto.dylib libcrypto.dylib

This way, at least in theory, when updating openssl the dylibs will always point to the latest versions. /usr/local/opt/openssl is actually a link to /usr/local/Cellar/openssl/Cellar/openssl/1.0.2t (the version of openssl installed by brew).

The reason the issue happens is actually explained by brew:

> openssl is keg-only, which means it was not symlinked into /usr/local, because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

Trying to run brew link openssl:

> Warning: Refusing to link macOS-provided software: openssl If you need > to have openssl first in your PATH run: echo 'export > PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile > > For compilers to find openssl you may need to set: export > LDFLAGS="-L/usr/local/opt/openssl/lib" export > CPPFLAGS="-I/usr/local/opt/openssl/include" > > For pkg-config to find openssl you may need to set: export > PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

So, basically you need to link them manually.

r.xuan from this Apple Dev thread identified the steps of a workaround for the error Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI. by replacing the libssl.dylib and libcrypto.dylib links in /usr/local/lib with links to libs from Homebrew's install of openssl.

The steps are:

Get fresh libs

1. brew update && brew upgrade && brew install openssl

2. cd /usr/local/Cellar/openssl/1.0.2t/lib

3. sudo cp libssl.1.0.0.dylib libcrypto.1.0.0.dylib /usr/local/lib/

Backup the old ones

4. cd /usr/local/lib

5. mv libssl.dylib libssl_bak.dylib

6. mv libcrypto.dylib libcrypto_bak.dylib

Create new links

7. sudo ln -s libssl.1.0.0.dylib libssl.dylib

8. sudo ln -s libcrypto.1.0.0.dylib libcrypto.dylib

I was seeing similar problems with ansible. The culprit was asn1crypto, and the problem has been already fixed.

My solution was to manually remove it and reinstall it with pip:

1. rm -r /usr/local/lib/python2.7/site-packages/asn1crypto*. This allowed pip to work without problems.
2. pip install asn1crypto, which installed 1.2.0:

  Found existing installation: asn1crypto 0.24.0
    Uninstalling asn1crypto-0.24.0:
      Successfully uninstalled asn1crypto-0.24.0
Successfully installed asn1crypto-1.2.0

NOTE: You can check if asn1crypto is the culprit by running python in verbose mode, e.g. python -v $(which ansible). In my case it crashed while doing some asn1crypto related imports:

# /usr/local/lib/python2.7/site-packages/asn1crypto/_perf/_big_num_ctypes.pyc matches /usr/local/lib/python2.7/site-packages/asn1crypto/_perf/_big_num_ctypes.py
import asn1crypto._perf._big_num_ctypes # precompiled from /usr/local/lib/python2.7/site-packages/asn1crypto/_perf/_big_num_ctypes.pyc
[1]    59247 abort      python -v $(which ansible)

Related: https://github.com/Homebrew/homebrew-core/issues/44996

It must be usage of some dependencies like cryptography

Solution:

cd your-site-packages-path/
vim ./asn1crypto/_int.py

find this line; delete it, and everything is ok

# from ._perf._big_num_ctypes import libcrypto

Here is my problem

Process:               Python [85179]
Path:                  /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.4 (3.7.4)
Code Type:             X86-64 (Native)
Parent Process:        ??? [85161]
Responsible:           iTerm2 [11711]
User ID:               501

Date/Time:             2019-10-07 23:00:25.143 +0800
OS Version:            Mac OS X 10.15 (19A582a)
Report Version:        12
Bridge OS Version:     3.0 (14Y906)
Anonymous UUID:        32C73ADD-1291-FA0E-DC02-48D539674325


Time Awake Since Boot: 42000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

Try:

python3 -m pip install oscrypto

Worked for me!

I’m afraid that none of these answers are acceptable to me. Because these answers didn’t fix the cause. Some of them are lucky, or misunderstand, or even wrong. So I’m going to provide my own solution and explain in details for the problem.

> Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

The root cause is quite clear itself. Your Python was trying to open (by dlopen) an unversioned OpenSSL shared library named libcrypto. Apple doesn’t allow anyone to use it for the security reason since Catalina. So the solution is straightforward. Just using versioning OpenSSL instead.

I write a python script named openlib.py to reproduce the problem.

import sys
from ctypes.util import find_library
from ctypes import CDLL

name = sys.argv[1]
path = find_library(name)
print(f"path: {path}")
lib = CDLL(path)

I use system Python for demo here

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.15.7
BuildVersion:   19H15
$ /usr/bin/python3 --version
Python 3.8.2
$ ls -al /usr/lib/ | grep 'libcrypto\|libssl'
.rwxr-xr-x 1.1M root 22 Sep  8:29 libcrypto.0.9.7.dylib
.rwxr-xr-x 1.4M root 22 Sep  8:29 libcrypto.0.9.8.dylib
.rwxr-xr-x 1.5M root 22 Sep  8:29 libcrypto.35.dylib
.rwxr-xr-x 1.5M root 22 Sep  8:29 libcrypto.41.dylib
.rwxr-xr-x 1.5M root 22 Sep  8:29 libcrypto.42.dylib
.rwxr-xr-x 1.5M root 22 Sep  8:29 libcrypto.44.dylib
.rwxr-xr-x  32k root 22 Sep  8:29 libcrypto.dylib
.rwxr-xr-x 212k root 22 Sep  8:29 libssl.0.9.7.dylib
.rwxr-xr-x 335k root 22 Sep  8:30 libssl.0.9.8.dylib
.rwxr-xr-x 330k root 22 Sep  8:28 libssl.35.dylib
.rwxr-xr-x 313k root 22 Sep  8:29 libssl.43.dylib
.rwxr-xr-x 300k root 22 Sep  8:29 libssl.44.dylib
.rwxr-xr-x 294k root 22 Sep  8:29 libssl.46.dylib
.rwxr-xr-x  32k root 22 Sep  8:29 libssl.dylib
$ /usr/bin/python3 openlib.py libcrypto
path: /usr/lib/libcrypto.dylib
Abort trap: 6
$ /usr/bin/python3 openlib.py libcrypto.35
path: /usr/lib/libcrypto.35.dylib
$ /usr/bin/python3 openlib.py libcrypto.44
path: /usr/lib/libcrypto.44.dylib

As you can see. Python has crashed with argument libcrypto as following diagnostic report. Look similar, right? Bingo!

Process:               Python [97291]
Path:                  /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.8.2 (3.8.2)
Build Info:            python3-73040006000000~117
Code Type:             X86-64 (Native)
Parent Process:        bash [84388]
Responsible:           iTerm2 [7705]
User ID:               501

Date/Time:             2020-12-26 00:28:00.281 +0800
OS Version:            Mac OS X 10.15.7 (19H15)
Report Version:        12
Anonymous UUID:        1C43F3DB-1783-4B94-B663-7F7E8D331B56


Time Awake Since Boot: 53000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
/usr/lib/libcrypto.dylib
abort() called
Invalid dylib load. Clients should not load the unversioned libcrypto dylib as it does not have a stable ABI.

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff69bba33a __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff69c76e60 pthread_kill + 430
2   libsystem_c.dylib             	0x00007fff69b41808 abort + 120
3   libcrypto.dylib               	0x00007fff6766b7e4 __report_load + 415

Based on the path of your Python /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python. Apparently, Your Python is installed by Homebrew. I believe Homebrew Python 3 has been linked with keg-only OpenSSL. So It must be some packages that using unversioned OpenSSL. Looking at file ctypes/macholib/dyld.py 2. It searches for any libraries in following directories in the specified order:

DEFAULT_LIBRARY_FALLBACK = [
    os.path.expanduser("~/lib"),
    "/usr/local/lib",
    "/lib",
    "/usr/lib",
]

$ /usr/bin/python3
Python 3.8.2 (default, Nov  4 2020, 21:23:28)
[Clang 12.0.0 (clang-1200.0.32.28)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes.util import find_library
>>> find_library('libcrypto')
'/usr/lib/libcrypto.dylib'
>>>

So the easy fix is linking versioning OpenSSL to the library path in your home directory.

$ pwd
/Users/gasolwu
$ ln -s /usr/lib/libcrypto.44.dylib $HOME/lib/libcrypto.dylib
$ $ ls ~/lib
libcrypto.dylib

After that. Test opening OpenSSL by running the provided script openlib.py. It returns library path successfully without crash.

$ /usr/bin/python3 openlib.py libcrypto
path: /Users/gasolwu/lib/libcrypto.dylib

I use Homebrew Python 3 too. So I fixed it a couple of days ago and have sent a pull request. If you have already upgraded to the latest version of Python and If PR has been merge and the bottle has built and published. Simply run the command with brew reinstall [email protected] will be the easiest way to fix your problem.

Don’t break your system by disabling SIP and using sudo to overwrite your system OpenSSL.

Don’t install duplicate libraries that waste your disk space. There is no need to install another OpenSSL in any location.

Don't use environment variable DYLD_LIBRARY_PATH as follows. You have to declare every time if you don't add this line to your configuration of the shell. If you do, It will affect every program on your machine.

export DYLD_LIBRARY_PATH=/usr/local/opt/openssl/lib:$DYLD_LIBRARY_PATH

Finally. If you fixed the problem by updating python dependencies. You might be lucky. Some packages have fixed the problem by looking for versioning OpenSSL. But many are not.

If you're using Kevlar from DevMate, upgrade to 4.3.1, which "Fixed macOS Catalina crash caused by version of libcrypto.dylib".

Looks like it was a Homebrew issue. I did brew reinstall python3 and it worked.

To follow the answers mentioned above, wanted to link libssl.dylib file but found is no such location as below:

/usr/local/Cellar/openssl/1.0.2t/lib/

However as the accepted answer by @bixel found the file in below location

/usr/local/opt/openssl/lib

and it worked for me.

I met the same issue when I was using ctypes.cdll to open /usr/lib/libcrypto.dylib with Python 3.7. However the dylib COULD be opened with Python 2.7.

I installed the latest openssl with brew install, then set the environment variables and created links as what they suggested above, NOTHING good was happened.

After several hours' digging, I found a workaround solution.

I found some libcrypto.X.dylib in /usr as following,

/usr/lib/libcrypto.dylib
/usr/lib/libcrypto.0.9.7.dylib
/usr/lib/libcrypto.0.9.8.dylib
/usr/lib/libcrypto.35.dylib
/usr/lib/libcrypto.41.dylib
/usr/lib/libcrypto.42.dylib
/usr/lib/libcrypto.44.dylib

/usr/local/opt/openssl/lib/libcrypto.1.1.dylib
/usr/local/opt/openssl/lib/libcrypto.dylib

Firstly, I used the followed one to replace that in /usr/lib instead.

os.environ['DYLD_FALLBACK_LIBRARY_PATH'] = '/usr/local/opt/openssl/lib'

It could be loaded but some apis were missing,

AttributeError: dlsym(0x..., ECDH_OpenSSL): symbol not found

I created a link for /usr/lib/libcrypto.X.dylib in my script path.

ln -s /usr/lib/libcrypto.X.dylib lib/libcrypto.dylib

Then add the path to DYLD_FALLBACK_LIBRARY_PATH

os.environ['DYLD_FALLBACK_LIBRARY_PATH'] = 'lib' # It should be a absolute path

At last, it worked.

Use the following steps to solve：

• brew update && brew upgrade && brew reinstall openssl
• cd /usr/local/Cellar/[email protected]/1.1.1g/lib
• sudo cp libssl.1.1.1.dylib libcrypto.1.1.1.dylib /usr/local/lib/
• sudo ln -s libssl.1.0.0.dylib libssl.dylib
• sudo ln -s libcrypto.1.0.0.dylib libcrypto.dylib`