Which TensorFlow and CUDA version combinations are compatible?

TensorflowCudaVersionCompatibilityCudnn

Tensorflow Problem Overview


I have noticed that some newer TensorFlow versions are incompatible with older CUDA and cuDNN versions. Does an overview of the compatible versions or even a list of officially tested combinations exist? I can't find it in the TensorFlow documentation.

Tensorflow Solutions


Solution 1 - Tensorflow

TL;DR) See this table: https://www.tensorflow.org/install/source#gpu

Generally:

Check the CUDA version:

cat /usr/local/cuda/version.txt

and cuDNN version:

grep CUDNN_MAJOR -A 2 /usr/local/cuda/include/cudnn.h

and install a combination as given below in the images or here.

The following images and the link provide an overview of the officially supported/tested combinations of CUDA and TensorFlow on Linux, macOS and Windows:

Minor configurations:

Since the given specifications below in some cases might be too broad, here is one specific configuration that works:

  • tensorflow-gpu==1.12.0
  • cuda==9.0
  • cuDNN==7.1.4

The corresponding cudnn can be downloaded here.

Tested build configurations

Please refer to https://www.tensorflow.org/install/source#gpu for a up-to-date compatibility chart (for official TF wheels).

(figures updated May 20, 2020)

Linux GPU

enter image description here

Linux CPU

enter image description here

macOS GPU

enter image description here

macOS CPU

enter image description here

Windows GPU

enter image description here

Windows CPU

enter image description here

Updated as of Dec 5 2020: For the updated information please refer Link for Linux and Link for Windows.

Solution 2 - Tensorflow

The compatibility table given in the tensorflow site does not contain specific minor versions for cuda and cuDNN. However, if the specific versions are not met, there will be an error when you try to use tensorflow.

For tensorflow-gpu==1.12.0 and cuda==9.0, the compatible cuDNN version is 7.1.4, which can be downloaded from here after registration.

You can check your cuda version using
nvcc --version

cuDNN version using
cat /usr/include/cudnn.h | grep CUDNN_MAJOR -A 2

tensorflow-gpu version using
pip freeze | grep tensorflow-gpu

UPDATE: Since tensorflow 2.0, has been released, I will share the compatible cuda and cuDNN versions for it as well (for Ubuntu 18.04).

  • tensorflow-gpu = 2.0.0
  • cuda = 10.0
  • cuDNN = 7.6.0

Solution 3 - Tensorflow

if you are coding in jupyter notebook, and want to check which cuda version tf is using, run the follow command directly into jupyter cell:

!conda list cudatoolkit

!conda list cudnn

and to check if the gpu is visible to tf:

tf.test.is_gpu_available(
    cuda_only=False, min_cuda_compute_capability=None
)

Solution 4 - Tensorflow

You can use this configuration for cuda 10.0 (10.1 does not work as of 3/18), this runs for me:

  • tensorflow>=1.12.0
  • tensorflow_gpu>=1.4

Install version tensorflow gpu:

pip install tensorflow-gpu==1.4.0

Solution 5 - Tensorflow

I had installed CUDA 10.1 and CUDNN 7.6 by mistake. You can use following configurations (This worked for me - as of 9/10). :

  • Tensorflow-gpu == 1.14.0
  • CUDA 10.1
  • CUDNN 7.6
  • Ubuntu 18.04

But I had to create symlinks for it to work as tensorflow originally works with CUDA 10.

sudo ln -s /opt/cuda/targets/x86_64-linux/lib/libcublas.so /opt/cuda/targets/x86_64-linux/lib/libcublas.so.10.0
sudo cp /usr/lib/x86_64-linux-gnu/libcublas.so.10 /usr/local/cuda-10.1/lib64/
sudo ln -s /usr/local/cuda-10.1/lib64/libcublas.so.10 /usr/local/cuda-10.1/lib64/libcublas.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcusolver.so.10 /usr/local/cuda/lib64/libcusolver.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcurand.so.10 /usr/local/cuda/lib64/libcurand.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcufft.so.10 /usr/local/cuda/lib64/libcufft.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda/lib64/libcudart.so.10.0
sudo ln -s /usr/local/cuda/targets/x86_64-linux/lib/libcusparse.so.10 /usr/local/cuda/lib64/libcusparse.so.10.0

And add the following to my ~/.bashrc -

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export PATH=/usr/local/cuda-10.1/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/targets/x86_64-linux/lib/

Solution 6 - Tensorflow

Thanks for the first answer.

Something about backward compatibility.

I can successfully install tensorflow-2.4.0 with cuda-11.1 and cudnn 8.0.5.

enter image description here

Solution 7 - Tensorflow

I had a similar problem after upgrading to TF 2.0. The CUDA version that TF was reporting did not match what Ubuntu 18.04 thought I had installed. It said I was using CUDA 7.5.0, but apt thought I had the right version installed.

What I eventually had to do was grep recursively in /usr/local for CUDNN_MAJOR, and I found that /usr/local/cuda-10.0/targets/x86_64-linux/include/cudnn.h did indeed specify the version as 7.5.0.
/usr/local/cuda-10.1 got it right, and /usr/local/cuda pointed to /usr/local/cuda-10.1, so it was (and remains) a mystery to me why TF was looking at /usr/local/cuda-10.0.

Anyway, I just moved /usr/local/cuda-10.0 to /usr/local/old-cuda-10.0 so TF couldn't find it any more and everything then worked like a charm.

It was all very frustrating, and I still feel like I just did a random hack. But it worked :) and perhaps this will help someone with a similar issue.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionwhiletrueView Question on Stackoverflow
Solution 1 - TensorflowwhiletrueView Answer on Stackoverflow
Solution 2 - TensorflowAtul BalajiView Answer on Stackoverflow
Solution 3 - TensorflowSavrigeView Answer on Stackoverflow
Solution 4 - TensorflowED-DOUGHMI younesView Answer on Stackoverflow
Solution 5 - Tensorflowyatish h rView Answer on Stackoverflow
Solution 6 - TensorflowoneView Answer on Stackoverflow
Solution 7 - TensorflowMatt GinsbergView Answer on Stackoverflow