ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Python Pandas Numpy Scikit Learn Python 3.7

Python Problem Overview

Importing from pyxdameraulevenshtein gives the following error, I have

pyxdameraulevenshtein==1.5.3, 
pandas==1.1.4 and 
scikit-learn==0.20.2. 
Numpy is 1.16.1. 
Works well in Python3.6, Issue in Python3.7.

Has anyone been facing similar issues with Python3.7 (3.7.9), docker image - python:3.7-buster

__init__.pxd:242: in init pyxdameraulevenshtein
    ???
E   ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Python Solutions

Solution 1 - Python

I'm in Python 3.8.5. It sounds too simple to be real, but I had this same issue and all I did was reinstall numpy. Gone.

pip install --upgrade numpy

pip uninstall numpy
pip install numpy

Solution 2 - Python

try with numpy==1.20.0 this worked here, even though other circumstances are different (python3.8 on alpine 3.12).

Solution 3 - Python

Indeed, (building and) installing with numpy>=1.20.0 should work, as pointed out e.g. by this answer below. However, I thought some background might be interesting -- and provide also alternative solutions.

There was a change in the C API in numpy 1.20.0. In some cases, pip seems to download the latest version of numpy for the build stage, but then the program is run with the installed version of numpy. If the build version used in <1.20, but the installed version is =>1.20, this will lead to an error.

(The other way around it should not matter, because of backwards compatibility. But if one uses an installed version numpy<1.20, they did not anticipate the upcoming change.)

This leads to several possible ways to solve the problem:

upgrade (the build version) to numpy>=1.20.0
use minmum supported numpy version in pyproject.toml (oldest-supported-numpy)
install with --no-binary
install with --no-build-isolation

For a more detailed discussion of potential solutions, see https://github.com/scikit-learn-contrib/hdbscan/issues/457#issuecomment-773671043.

Solution 4 - Python

Solution without upgrading `numpy`

While upgrading the numpy version would often solve the issue, it's not always viable. Good example is the case when you're using tensorflow==2.6.0 which isn't compatible with the newest numpy version (it requires ~=1.19.2).

As already mentioned in FZeiser's answer, there was a change in numpys C API in version 1.20.0. There are packages that rely on this C API when they are being built, e.g. pycocotools. Given that pips dependency resolver doesn't guarantee any order for installing the packages, the following might happen:

pip figures out that it needs to install numpy and it chooses the latest version, 1.21.2 as of the time writing this answer.
It then builds a package that depends on numpy and its C API, e.g. pycocotools. This package is now compatible with numpy 1.21.2 C API.
At a later point pip needs to install a package that has a requirement for an older version of numpy, e.g. tensorflow==2.6.0 which would try to install numpy==1.19.5. As a result, numpy==1.21.2 is uninstalled and the older version is installed.
When running code that uses pycocotools, its current installation relies on the updated numpy C API, yet the numpy version was downgraded which would result in the error.

Solution

You should rebuild the package with the outdated numpy C API usage, e.g. for pycocotools:

pip uninstall pycocotools
pip install pycocotools --no-binary pycocotools

Solution 5 - Python

I had this issue when using the tensorflow object api. Tensorflow is currently NOT compatible with numpy==1.20 (although this issue is not apparent until later). In my case, the issue was caused by pycocotools. I fixed by installing an older version.

pip install pycocotools==2.0.0

Solution 6 - Python

Upgrading python version to numpy ==1.21.1 worked for me!

Solution 7 - Python

what worked for me was:

pip uninstall numpy
conda install -y -c conda-forge numpy

as bizzare as it might sound...I didn't even have to uninstall it with conda which seemed odd to me. I am using python 3.9

Solution 8 - Python

For anyone using Poetry it is necessary to have experimental.new-installer set to true for an application with a numpy<1.20 dependency to be built correctly i.e:

poetry config experimental.new-installer true

It is true by default but if (as was the case for me) it has been changed it can catch you out.

My application uses Tensorflow and I did not therefore have the option of upgrading to >1.20. Poetry also does not support --no-binary dependencies.

Solution 9 - Python

numpy version 1.22 solved it for me.

Solution 10 - Python

After you pip install any package, makes sure you restart the Kernel and should work. usually packages get upgraded automatically and all you need is a quick restart. At least, this what worked in my situation and I was getting the same error when I tried to install and use pomegranate.

Solution 11 - Python

I was facing same issue in raspberry pi 3. Actually the error is with pandas. Although tensorflow need numpy~=1.19.2 , but pandas is not compliable with it. So, I have upgraded (because downgrading is not) my numpy to latest version and all works fine!!!!.

root@raspberrypi:/home/pi# python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np 
>>> np.__version__
'1.21.5'
>>> import pandas as pd
>>> pd.__version__
'1.3.5'
>>> import tensorflow as tf
>>> tf.__version__
'2.4.0'
>>> tf.keras.__version__
'2.4.0'
>>> tf.keras.layers
<module 'tensorflow.keras.layers' from '/usr/local/lib/python3.7/dist-packages/tensorflow/keras/layers/__init__.py'>

Same issue here also - https://github.com/bitsy-ai/tensorflow-arm-bin/issues/5

Tensorflow source: https://github.com/bitsy-ai/tensorflow-arm-bin

Solution 12 - Python

Use python virtual environments and install gensim using :

pip install gensim==3.8.3

Solution 13 - Python

Upgrade numpy version:

pip install -U numpy

Solution 14 - Python

This worked for me (when nothing else on this page did):

# Create environment with conda or venv.
# Do *not* install any other packages here.
pip install numpy=1.21.5
# Install all other packages here.
# This works as a package may build against the currently installed version of numpy.

This solved a particularly brutal issue that was unresolvable by all other answers on this page as of 2022-04-11:

> Other answers try to fix the problem after it occurred, this fixes the problem before it occurs.

In addition, experiment with different versions of Python, e.g. 3.8, 3.9, 3.10.

Reference: Excellent answer by @FZeiser that explains why this works.

Solution 15 - Python

I encountered with the same problem with python3.10.4,numpy1.21.5, I solved it only after I updated numpy to 1.22.3 via pip uninstall numpy and pip install numpy. Only pip install --upgrade numpy didn't work.

> PS D:\quant\vnpy-master\examples\veighna_trader> python .\run.py > Traceback (most recent call last): File > "D:\quant\vnpy-master\examples\veighna_trader\run.py", line 31, in > from vnpy_optionmaster import OptionMasterApp File > "D:\it_soft\python3.10.4\Lib\site-packages\vnpy_optionmaster__init__.py", > line 26, in from .engine import OptionEngine, APP_NAME File > "D:\it_soft\python3.10.4\Lib\site-packages\vnpy_optionmaster\engine.py", > line 34, in from .pricing import binomial_tree_cython as > binomial_tree File "binomial_tree_cython.pyx", line 1, in init > binomial_tree_cython ValueError: numpy.ndarray size changed, may > indicate binary incompatibility. Expected 96 from C header, got 88 > from PyObject

Solution 16 - Python

For almost the same image : python:3.7-slim-buster

I started to have this problem just today, it was non exitent before.

I solved it by removing numpy from requirement.txt file and doing instead the following in my Dockerfile:

RUN pip3 install --upgrade  --no-binary numpy==1.18.1 numpy==1.18.1 \
&& pip3 install -r requirements.txt

I use some old versions of keras and its librairies and updrading to numpy 1.20.0 didn't work for those librairies. But I think the solution consist in the first command that I gave you wich tell pip to try to not compile numpy and download a pre-compiled version.

The trick in the command is that you might find people telling you to use --no-binary option of pip to solve the problem, but they don't specify how and it might be tricky (as it happened to me); you have to write the package two times in the command in order for it to work or else pip will throw you an error.

I think the --upgrade option in the first command isn't necessary.

Solution 17 - Python

Install older version of gensim, It works!

pip install gensim==3.5.0

conda install gensim==3.5.0

Content Type	Original Author	Original Content on Stackoverflow
Question	Sachit Jani	View Question on Stackoverflow
Solution 1 - Python	Andreu	View Answer on Stackoverflow
Solution 2 - Python	mmacata	View Answer on Stackoverflow
Solution 3 - Python	FZeiser	View Answer on Stackoverflow
Solution 4 - Python	Ivaylo Toskov	View Answer on Stackoverflow
Solution 5 - Python	Royce Schultz	View Answer on Stackoverflow
Solution 6 - Python	Nsaiyed	View Answer on Stackoverflow
Solution 7 - Python	Charlie Parker	View Answer on Stackoverflow
Solution 8 - Python	elukem	View Answer on Stackoverflow
Solution 9 - Python	Shubham	View Answer on Stackoverflow
Solution 10 - Python	Tony Bazzi	View Answer on Stackoverflow
Solution 11 - Python	MegaCreater	View Answer on Stackoverflow
Solution 12 - Python	Hamidreza-Safari	View Answer on Stackoverflow
Solution 13 - Python	JKO	View Answer on Stackoverflow
Solution 14 - Python	Contango	View Answer on Stackoverflow
Solution 15 - Python	Andy Kong	View Answer on Stackoverflow
Solution 16 - Python	H4k333m	View Answer on Stackoverflow
Solution 17 - Python	rohan goli	View Answer on Stackoverflow

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

Solution 4 - Python

Solution without upgrading `numpy`

Solution

Solution 5 - Python

Solution 6 - Python

Solution 7 - Python

Solution 8 - Python

Solution 9 - Python

Solution 10 - Python

Solution 11 - Python

Solution 12 - Python

Solution 13 - Python

Solution 14 - Python

Solution 15 - Python

Solution 16 - Python

Solution 17 - Python

When to use record vs class vs struct

SQL Group By with an Order By

Attributions

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

Solution 4 - Python

Solution without upgrading numpy

Solution

Solution 5 - Python

Solution 6 - Python

Solution 7 - Python

Solution 8 - Python

Solution 9 - Python

Solution 10 - Python

Solution 11 - Python

Solution 12 - Python

Solution 13 - Python

Solution 14 - Python

Solution 15 - Python

Solution 16 - Python

Solution 17 - Python

When to use record vs class vs struct

SQL Group By with an Order By

Attributions

Solution without upgrading `numpy`