Python3: ImportError: No module named '_ctypes' when using Value from module multiprocessing

I am using Ubuntu and have installed Python 2.7.5 and 3.4.0. In Python 2.7.5 I am able to successfully assign a variable x = Value('i', 2), but not in 3.4.0. I am getting:

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/local/lib/python3.4/multiprocessing/context.py", line 132, in Value
      from .sharedctypes import Value
   File "/usr/local/lib/python3.4/multiprocessing/sharedctypes.py", line 10, in <
module>
   import ctypes
   File "/usr/local/lib/python3.4/ctypes/__init__.py", line 7, in <module>
      from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'

I just updated to 3.3.2 through installing the source of 3.4.0. It installed in /usr/local/lib/python3.4.

Did I update to Python 3.4 correctly?

One thing I noticed that Python 3.4 is installed in usr/local/lib, while Python 3.3.2 is still installed in usr/lib, so it was not overwritten.

Installing libffi-dev and re-installing python3.7 fixed the problem for me.

to cleanly build py 3.7 libffi-dev is required or else later stuff will fail

If using RHEL/Fedora:

yum install libffi-devel

or

sudo dnf install libffi-devel

If using Debian/Ubuntu:

sudo apt-get install libffi-dev

On a fresh Debian image, cloning https://github.com/python/cpython and running:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install build-essential python-dev python-setuptools python-pip python-smbus
sudo apt-get install libncursesw5-dev libgdbm-dev libc6-dev
sudo apt-get install zlib1g-dev libsqlite3-dev tk-dev
sudo apt-get install libssl-dev openssl
sudo apt-get install libffi-dev

Now execute the configure file cloned above:

./configure
make # alternatively `make -j 4` will utilize 4 threads
sudo make altinstall

Got 3.7 installed and working for me.

SLIGHT UPDATE

Looks like I said I would update this answer with some more explanation and two years later I don't have much to add.

• this SO post explains why certain libraries like python-dev might be necessary.
• this SO post explains why one might use the altinstall as opposed to install argument in the make command.

Aside from that I guess the choice would be to either read through the cpython codebase looking for #include directives that need to be met, but what I usually do is keep trying to install the package and just keep reading through the output installing the required packages until it succeeds.

Reminds me of the story of the Engineer, the Manager and the Programmer whose car rolls down a hill.

If you use pyenv and get error "No module named '_ctypes'" (like i am) on Debian/Raspbian/Ubuntu you need to run this commands:

sudo apt-get install libffi-dev
pyenv uninstall 3.7.6
pyenv install 3.7.6

Put your version of python instead of 3.7.6

Detailed steps to install Python 3.7 in CentOS or any redhat linux machine:

1. Download Python from https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
2. Extract the content in new folder
3. Open Terminal in the same directory
4. Run below code step by step :

> sudo yum -y install gcc gcc-c++ > sudo yum -y install zlib zlib-devel > sudo yum -y install libffi-devel > ./configure > make > make install

Thought I'd add the Centos installs:

sudo yum -y install gcc gcc-c++ 
sudo yum -y install zlib zlib-devel
sudo yum -y install libffi-devel 

Check python version:

python3 -V

Create virtualenv:

virtualenv -p python3 venv

This solved the same error for me on Debian:

sudo apt-get install libffi-dev

and compile again

Reference: issue31652

None of the solution worked. You have to recompile your python again; once all the required packages were completely installed.

Follow this:

1. Install required packages
2. Run ./configure --enable-optimizations

https://gist.github.com/jerblack/798718c1910ccdd4ede92481229043be

On my Ubuntu 18.04 machine, I had the common problem of python not finding _ctypes with the pyenv installed python.

In my case libffi-dev was already installed. Installing cpython from source, as suggested by @MikeiLL, didn't help either.

Turned out to be an homebrew issue.

ajkerrigans suggested solution on pyenvs github issues solved this problem for me.

Solution summary: Tell pyenv to build Python using the Homebrew-managed GCC, with a command like:

CC="$(brew --prefix gcc)/bin/gcc-11" \
pyenv install --verbose 3.10.0

This assumes that any build dependencies have also been installed via Homebrew as specified in the pyenv wiki. As of this writing, that looks like this for Homebrew on Linux:

brew install bzip2 libffi libxml2 libxmlsec1 openssl readline sqlite xz zlib

I run into this error when I tried to install Python 3.7.3 in Ubuntu 18.04 with next command: $ pyenv install 3.7.3. Installation succeeded after running $ sudo apt-get update && sudo apt-get install libffi-dev (as suggested here). The issue was solved there.

Based on this answer, just copy-paste into the terminal.

First run: sudo apt-get -y update

then:

sudo apt-get -y upgrade
sudo apt-get -y dist-upgrade
sudo apt-get -y install build-essential python-dev python-setuptools python-pip python-smbus
sudo apt-get -y install libncursesw5-dev libgdbm-dev libc6-dev
sudo apt-get -y install zlib1g-dev libsqlite3-dev tk-dev
sudo apt-get -y install libssl-dev openssl
sudo apt-get -y install libffi-dev

PS: You can just copy-paste the whole chunk into the terminal in one go.

In my case what was causing all sorts of Python installation issues including the one having to do with _ctypes and libffi was Homebrew on Linux / Linuxbrew. pyenv was happy again once brew was no longer in the $PATH.

Refer to this thread or this thread, for customized installation of libffi, it is difficult for Python3.7 to find the library location of libffi. An alternative method is to set the CONFIGURE_LDFLAGS variable in the Makefile, for example CONFIGURE_LDFLAGS="-L/path/to/libffi-3.2.1/lib64".

My solution: Installing libffi-dev with apt-get didn't help. But this helped: Installing libffi from source and then installing Python 3.8 from source.

My configuration: Ubuntu 16.04 LTS Python 3.8.2

Step by step:

I got the error message "ModuleNotFoundError: No module named '_ctypes'" when starting the debugger from Visual Studio Code, and when running python3 -c "import sklearn; sklearn.show_versions()".

• download libffi v3.3 from https://github.com/libffi/libffi/releases
• install libtool: sudo apt-get install libtool The file README.md from libffi mentions that autoconf and automake are also necessary. They were already installed on my system.
• configure libffi without docs:

./configure --disable-docs

make check

sudo make install

• download python 3.8 from https://www.python.org/downloads/
• ./configure
• make
• make test
• make install

After that my python installation could find _ctypes.

CentOS without root

1. Install libffi-3.2 (Do NOT use libffi-3.3)

   wget ftp://sourceware.org/pub/libffi/libffi-3.2.tar.gz
   tar -xzf libffi-3.2.tar.gz
   cd libffi-3.2/
   ./configure --prefix=$YOUR_LIBFFI_DIR
   make && make install

2. Install Python3

   ./configure --prefix=$YOUR_PATH/python/3.7.10 LDFLAGS=-L${YOUR_LIBFFI_DIR}/lib64 PKG_CONFIG_PATH=${YOUR_LIBFFI_DIR}/lib/pkgconfig --enable-shared

   make && make install

Thanks for JohnWSteill

I was having the same problem. None of the above solutions worked for me. The key challenge was that I didn't have the root access. So, I first download the source of libffi. Then I compiled it with usual commands:

./configure --prefix=desired_installation_path_to_libffi
make 

Then I recompiled python using

./configure --prefix=/home/user123/Softwares/Python/installation3/  LDFLAGS='-L/home/user123/Softwares/library/libffi/installation/lib64'
make 
make install

In my case, 'home/user123/Softwares/library/libffi/installation/lib64' is path to LIBFFI installation directory where libffi.so is located. And, /home/user123/Softwares/Python/installation3/ is path to Python installation directory. Modify them as per your case.

If you don't mind using Miniconda, the necessary external libraries and _ctypes are installed by default. It does take more space and may require using a moderately older version of Python (e.g. 3.7.6 instead of 3.8.2 as of this writing).

You have to load the missing php3 (Python3) modules from the package manager. If you have Ubuntu I recommend the Synaptic Package Manager:

sudo apt-get install synaptic

There you can simply search for the missing modules. search for ctypes and install all the packages. Then go to your Python dir and do

./configure
make install.

This should solve your problem.

How to install Python from source without libffi in /usr/local?

1. Download libffi from github and install to /path/to/local
2. Download python source code and compile with the following configuration:

export PKG_CONFIG_PATH=/path/to/local/lib/pkgconfig
./configure --prefix=/path/to/python \
    LDFLAGS='-L/path/to/local/lib -Wl,-R/path/to/local/lib' \
    --enable-optimizations
make
make install

If you are doing something nobody here will listen you about because "you're doing it the wrong way", but you have to do it "the wrong way" for reasons too asinine to explain and also beyond your ability to control, you can try this:

Get libffi and install it into your user install area the usual way.

git clone https://github.com/libffi/libffi.git
cd libffi
./configure --prefix=path/to/your/install/root
make
make install

Then go back to your Python 3 source and find this part of the code in setup.py at the top level of the python source directory

        ffi_inc = [sysconfig.get_config_var("LIBFFI_INCLUDEDIR")]
        if not ffi_inc or ffi_inc[0] == '':
            ffi_inc = find_file('ffi.h', [], inc_dirs)
        if ffi_inc is not None:
            ffi_h = ffi_inc[0] + '/ffi.h'
            if not os.path.exists(ffi_h):
                ffi_inc = None
                print('Header file {} does not exist'.format(ffi_h))
        ffi_lib = None
        if ffi_inc is not None:
            for lib_name in ('ffi', 'ffi_pic'):
                if (self.compiler.find_library_file(lib_dirs, lib_name)):
                    ffi_lib = lib_name
                    break

        ffi_lib="ffi"  # --- AND INSERT THIS LINE HERE THAT DOES NOT APPEAR ---
        if ffi_inc and ffi_lib:
            ext.include_dirs.extend(ffi_inc)
            ext.libraries.append(ffi_lib)
            self.use_system_libffi = True

and add the line I have marked above with the comment. Why it is necessary, and why there is no way to get configure to respect '--without-system-ffi` on Linux platforms, perhaps I will find out why that is "unsupported" in the next couple of hours, but everything has worked ever since. Otherwise, best of luck... YMMV.

WHAT IT DOES: just overrides the logic there and causes the compiler linking command to add "-lffi" which is all that it really needs. If you have the library user-installed, it is probably detecting the headers fine as long as your PKG_CONFIG_PATH includes path/to/your/install/root/lib/pkgconfig.