How to free disk space taken up by (ana)conda?

PythonAnacondaConda

Python Problem Overview


I am using the conda package manager - a lot. By now I have quite a few environments and a lot of downloaded packages taking a lot of space on my SSD. An obvious path to free some of that space is to use the command

conda env export > environment.yml

from https://conda.io/docs/user-guide/tasks/manage-environments.html#exporting-the-environment-file to export which packages my old, inactive projects use(d) and then delete these environments. As far as I understand, this should free some of the space in anaconda2/envs/, but not in anaconda2/pkgs/. How do I get rid of these packages? Also, I suspect that there might be quite a few packages still sitting around, to which no environment is linking to - could that happen?

Questions:

  1. In general: What is the best way to reduce the space taken up by conda?
  2. How do I get rid of packages that no environment is using anymore? How do I prune my packages? I am searching for something like sudo apt-get autoremove from Ubuntu/Debian.

Python Solutions


Solution 1 - Python

You can free some space with:

conda clean --all

> clean Remove unused packages and caches.

Conda already use symlinks when possible for packages. So, not much to improve here, I guess.

> Ok, thanks, but I would like to know "not for a specific environment, but in general" - for all environments.

You can list all packages in all envs with a few lines of Python:

import os
import subprocess
for env in os.listdir('/Users/me/miniconda3/envs'):
    subprocess.call(['conda', 'list', '-n', env])

Solution 2 - Python

Finally I got around dealing with this issue. In the end it was a couple of days work:

  1. For all my Python projects I use PyCharm and with it I checked which project uses which environment. For all environments I used the conda env export > environment.yml to save the settings of the environment from https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#exporting-the-environment-file
  2. Check whether my projects still work with new environments created from the environment.yml.
  3. Use anaconda-clean from option B in https://docs.anaconda.com/anaconda/install/uninstall and put the created backup in a save place.
  4. Rename the old anaconda2 directory to anaconda2_backup.
  5. Install a new conda environment - miniconda3 in my case.
  6. Build new environments which are need for current projects from the environment.ymls and check whether these work.
  7. Delete the old anaconda backups.

Finally I also reduced my logical volume with https://blog.shadypixel.com/how-to-shrink-an-lvm-volume-safely/ but this is only for Linux users using LVMs.

This way I was able to free 20 to 30 GB of space.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMake42View Question on Stackoverflow
Solution 1 - PythonMike MüllerView Answer on Stackoverflow
Solution 2 - PythonMake42View Answer on Stackoverflow