How to find a Python package's dependencies
PythonPipPython Problem Overview
How can you programmatically get a Python package's list of dependencies?
The standard setup.py
has these documented, but I can't find an easy way to access it from either Python or the command line.
Ideally, I'm looking for something like:
$ pip install somepackage --only-list-deps
kombu>=3.0.8
billiard>=3.3.0.13
boto>=2.26
or:
>>> import package_deps
>>> package = package_deps.find('somepackage')
>>> print package.dependencies
['kombu>=3.0.8', 'billiard>=3.3.0.13', 'boto>=2.26']
Note, I'm not talking about importing a package and finding all referenced modules. While this might find most of the dependent packages, it wouldn't be able to find the minimum version number required. That's only stored in the setup.py.
Python Solutions
Solution 1 - Python
In addition to the pip show [package name]
command, there is pipdeptree
.
Just do
$ pip install pipdeptree
then run
$ pipdeptree
and it will show you your dependencies in a tree form, e.g.,
flake8==2.5.0
- mccabe [required: >=0.2.1,<0.4, installed: 0.3.1]
- pep8 [required: !=1.6.0,>=1.5.7,!=1.6.1,!=1.6.2, installed: 1.5.7]
- pyflakes [required: >=0.8.1,<1.1, installed: 1.0.0]
ipdb==0.8
- ipython [required: >=0.10, installed: 1.1.0]
The project is located at https://github.com/naiquevin/pipdeptree, where you will also find usage information.
Solution 2 - Python
Try to use show
command in pip
, for example:
$ pip show tornado
---
Name: tornado
Version: 4.1
Location: *****
Requires: certifi, backports.ssl-match-hostname
Update (retrieve deps with specified version):
from pip._vendor import pkg_resources
_package_name = 'somepackage'
_package = pkg_resources.working_set.by_key[_package_name]
print([str(r) for r in _package.requires()]) # retrieve deps from setup.py
Output: ['kombu>=3.0.8',
'billiard>=3.3.0.13',
'boto>=2.26']
Solution 3 - Python
Brief summary about the methods that are found and tested on the Windows machine:
-
Parse the json file of PyPI:
https://pypi.org/pypi/<package>/<version>/json
(#7) -
Check
/site-packages/<package-version>.dist-info/METADATA
(#13, pre-installation required) -
pip install --no-install <package>
: deprecated (#11) -
pip install --download <package>
: deprecated (#12) -
Write a script with
import pip
andpip._vendor.pkg_resources
: deprecated (#6) -
Write a script with
import pkg_resources
ofsetuptools
package (#4, pre-installation required) -
Check https://libraries.io/ (#5)
-
Use
Johnnydep
package (#10): Test hangs in many cases. -
conda info [package_name]
: deprecated (#9) -
conda search [package_name] --info
- Additional packages can be listed:
vc2015_runtime
,python_abi
,libflang
, and etc. - Note that this method can list different packages depending on the build and channel.
e.g.conda search "Django==3.2" --info -c conda-forge
- Additional packages can be listed:
django 3.2 pyhd3eb1b0_0
-----------------------
file name : django-3.2-pyhd3eb1b0_0.conda
...
timestamp : 2021-04-06 20:19:41 UTC
dependencies:
- asgiref
- psycopg2
- python
- pytz
- sqlparse
django 3.2 pyhd8ed1ab_0
-----------------------
file name : django-3.2-pyhd8ed1ab_0.tar.bz2
...
timestamp : 2021-04-07 21:15:25 UTC
dependencies:
- asgiref >=3.3.2,<4
- python >=3.6
- pytz
- sqlparse >=0.2.2
One more thing to note is that each method can provide a different result.
For example, requests/setup.py
identifies chardet
, idna
, urllib3
, and certifi
are required. Moreover, extra packages pyOpenSSL
, cryptography
, socks
, PySocks
, win-inet-pton
can be needed.
- Methods 1 and 2 agree with it.
- Method 8 just list all of them (pressing the explore dependencies button just hangs).
- Method 12 just list
chardet
,idna
,urllib3
, andcertifi
. - Methods 5 and 7 does not list any dependency if
requests
is installed usingpip
in the Linux docker. - Methods 5, 7, 9, and 10 list
chardet
,idna
,urllib3
, andcertifi
ifrequests
is installed in the conda environment of the Windows machine.
Solution 4 - Python
Quite a few answers here show pip being imported for use in programs. The documentation for pip strongly advises against this usage of pip.
Instead of accessing pkg_resources
via the pip import, you can actually just import pkg_resources
directly and use the same logic (which is actually one of the suggested solutions in the pip docs linked for anyone wanting to see package meta information programmatically) .
import pkg_resources
_package_name = 'yourpackagename'
def get_dependencies_with_semver_string():
package = pkg_resources.working_set.by_key[_package_name]
return [str(r) for r in package.requires()]
If you're having some trouble finding out exactly what your package name is, the WorkingSet
instance returned by pkg_resources.working_set
implements __iter__
so you can print all of them and hopefully spot yours in there :)
i.e.
import pkg_resources
def print_all_in_working_set():
ws = pkg_resources.working_set
for package_metadata in ws:
print(package_metadata)
This works with both python 2 and 3 (though you'll need to adjust the print statements for python2).
Solution 5 - Python
Use https://libraries.io/. It is a good place to explore dependencies before installing using pip.
Eg. Type google-cloud-storage and search, then you can find the page for the library (https://libraries.io/rubygems/google-cloud-storage). Select the version for which you want to explore the dependencies from the 'Releases' (default is the latest), Under 'Dependencies' you can find the dependency list and their supported versions.
Solution 6 - Python
The standard library offers importlib.metadata.requires()
since version 3.8 and onwards:
In [1]: from importlib.metadata import requires
In [2]: requires('pytype')
Out[2]:
['attrs (>=21.2.0)', 'importlab (>=0.6.1)', 'libcst', 'ninja (>=1.10.0.post2)', 'pyyaml (>=3.11)', 'six', 'tabulate', 'toml', 'typed-ast (>=1.4.3)', 'dataclasses ; python_version < "3.7"']
For older Python versions, there's the importlib-metadata
available. Related section in Python docs: Distribution requirements.
If you need to parse the strings returned by requires()
, I strongly suggest to use packaging
library since its packaging.requirements
module is a reference implementation of PEP 508. Example with a complex dependency specification string:
In [3]: from packaging.requirements import Requirement
In [4]: spec = 'requests [security,test] (>=2.21.0) ; implementation_name == "cpython"'
In [5]: r = Requirement(spec)
In [6]: r.name
Out[6]: 'requests'
In [7]: r.specifier
Out[7]: <SpecifierSet('>=2.21.0')>
In [8]: r.extras
Out[8]: {'security', 'test'}
In [9]: r.marker
Out[9]: <Marker('implementation_name == "cpython"')>
Solution 7 - Python
(THIS IS A LEGACY ANSWER AND SHOULD BE AVOIDED FOR MODERN PIP VERSIONS AND LEFT HERE FOR REFERENCE TO OLDER PIP VERSIONS ) Alex's answer is good (+1). In python:
pip._vendor.pkg_resources.working_set.by_key['twisted'].requires()
should return something like
[Requirement.parse('zope.interface>=3.6.0')]
where twisted is the name of the package, which you can find in the dictionary :
pip._vendor.pkg_resources.WorkingSet().entry_keys
to list them all:
dict = pip._vendor.pkg_resources.WorkingSet().entry_keys
for key in dict:
for name in dict[key]:
req =pip._vendor.pkg_resources.working_set.by_key[name].requires()
print('pkg {} from {} requires {}'.format(name,
key,
req))
should give you lists like this:
pkg pyobjc-framework-syncservices from /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC requires [Requirement.parse('pyobjc-core>=2.5.1'), Requirement.parse('pyobjc-framework-Cocoa>=2.5.1'), Requirement.parse('pyobjc-framework-CoreData>=2.5.1')]
Solution 8 - Python
There is the nice tool johnnydep. The benefit of it is that the package to inspect does not have to be installed.
pip install johnnydep
Use it like:
johnnydep deepspeech-tflite
Solution 9 - Python
Try this according to this article in python:
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
It will show like:
['behave==1.2.4', 'enum34==1.0', 'flask==0.10.1', 'itsdangerous==0.24', 'jinja2==2.7.2', 'jsonschema==2.3.0', 'markupsafe==0.23', 'nose==1.3.3', 'parse-type==0.3.4', 'parse==1.6.4', 'prettytable==0.7.2', 'requests==2.3.0', 'six==1.6.1', 'vioozer-metadata==0.1', 'vioozer-users-server==0.1', 'werkzeug==0.9.4']