UserWarning: Could not import the lzma module. Your installed Python is incomplete

After Installing Google Cloud Bigquery Module, if I import the module into python code. I see this warning message. Happening to me in python 3.7.3 Virtualenv.

Tried to reinstall GCP bigquery module Expectation-in python code if we write" from google.cloud import bigquery ".Should not result in any error or messege.

import os
import sys
import logging
from datetime import datetime
from google.cloud import bigquery

/home/informatica/.local/lib/python3.7/site-packages/pandas/compat/__init__.py:84: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
  warnings.warn(msg)
 exit()

If you compile Python from source, you must have the lzma-dev package installed, or it will not be built into python.

For ubuntu: sudo apt-get install liblzma-dev

For centos: yum install -y xz-devel

Then configure && make && make install

I used other good answers from here and didn't solve the problem (Ubuntu 18.04, Python3.8), still get this warning. Actually there is one more package is needed to be installed to solve the problem:

sudo apt-get install lzma

So the whole pipeline (run in the python source code folder):

sudo apt-get install liblzma-dev
sudo apt-get install lzma
./configure --enable-optimizations
sudo make
sudo make altinstall

On MacOS and pyenv (https://realpython.com/intro-to-pyenv/), I was able to have this warning go away by having xz installed with homebrew. Using version python 3.6.9 as an example

brew install xz && pyenv install 3.6.9

To use installed python, one needs to add this into .bash_profile

eval "$(pyenv init -)"

and start using it by running

pyenv global 3.6.9

On macOS, if you manage your python with pyenv and package with homebrew, you need to install "xz" first:

brew install xz

After installing xz, you can install python 3.8 by (I'm using 3.8.2 as an example:

pyenv install 3.8.2

Above will fix the problem.

If you are using centos and compile python from source, you can install from following commands

For centos: sudo yum install -y xz-devel

Recompile python from source code

cd Python-3.8*/
./configure --enable-optimizations
sudo make altinstall

This solution worked on my setup (Apple M1 with pyenv):

CFLAGS="-I$(brew --prefix xz)/include" LDFLAGS="-L$(brew --prefix xz)/lib" pyenv install 3.9.4

See last comment on https://github.com/pandas-dev/pandas/issues/27532

brew install xz # To pick up liblzma
prefix=$(brew --prefix)
export LDFLAGS="-L$prefix/opt/xz/lib $LDFLAGS"
export CPPFLAGS="-I$prefix/opt/xz/include $CPPFLAGS"
export PKG_CONFIG_PATH="$prefix/opt/xz/lib/pkgconfig:$PKG_CONFIG_PATH"
# YOU CANNOT HAVE THE GNUBINS in your PATH when you run this
PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.9.2
python3 -c "import lzma" # should work and not throw "cannot import _lzma"

What solved for me:

sudo apt-get install libbz2-dev
sudo cp /usr/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so/usr/local/lib/python3.8/
sudo cp /usr/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so /usr/local/lib/python3.8/

I did brew install xz and reinstalled everything, but that didn't do it for me.

What helped me was to add correct linkage for xz as well:

export LDFLAGS="-L$(brew --prefix xz)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib"
export CPPFLAGS="-I$(brew --prefix xz)/include  -I$(brew --prefix readline)/include -I$(brew --prefix zlib)/include -I$(xcrun --show-sdk-path)/usr/include"

I found the solution from: https://github.com/pandas-dev/pandas/issues/28219

I just ran: CPPFLAGS="-I$(brew --prefix xz)/include" pyenv install 3.10.0

• OS: Monterey
• M1 chip
• pyenv
• python 3.10.0

I see that

yum install -y lzma

Also runs without errors.

My OS is : CentOS 8.X

Step to step run command below, then fix the problem:

1. sudo yum install -y xz-devel
2. cd Python-3.8.5
3. sudo ./configure --prefix=/usr/local/python3.8.5 --enable-optimizations --with-ssl
4. sudo make
5. sudo make install

I had built Python 3.8 from source on Debian 10 and some times couldn't start mlflow server at all and sometimes got warnings albeit successful launches and also every pandas import gave me this warning.

Here's what worked for me:

1. purged the existing installation.
2. did sudo apt install libncurses-dev libgdbm-dev libz-dev tk-dev libsqlite3-dev libreadline-dev liblzma-dev libffi-dev libssl-dev
3. built python from source again.

I never got the warning again and had no problems whatsoever.