How to get a list of built-in modules in python?

Python

Python Problem Overview


I would like to get a list of names of built-in modules in python such that I can test the popularity of function's naming conventions (underline, CamelCase or mixedCase).

I know there is a Global Module Index but I am wondering if there is a list of strings, which is easier to use :)

Update:

len(dir(__builtins__)) = 145  
len(stdlib_list("2.7")) = 430  
help('modules') = 508 # counting manually the output

Python Solutions


Solution 1 - Python

The compiled-in module names are in sys.builtin_module_names. For all importable modules, see pkgutil.iter_modules.

Run these in a clean virtualenv to get (almost) only the modules that come with Python itself.


Note that a “popularity poll” will necessarily include modules that use old, discouraged naming conventions because they were written before today's guidelines were put in place, and can't change because need to be backwards compatible. It might be useful for something, but not for answering best-practice questions such as “How should I name my functions?”. For that, see the PEP8, the Python style guide, especially the “Naming Conventions” section.

Solution 2 - Python

How about this? Though, this gets a list of built-in functions and variables rather than modules...

dir(__builtins__)

help('modules') will give you a list of all modules, according to https://stackoverflow.com/questions/739993/unable-to-get-a-list-of-installed-python-modules. Not a list of strings, though.

Solution 3 - Python

Now there is a 3rd party package for this. It scrapes the TOC of the Standard Library page in the official Python docs and builds a list.

You can install it using pip

pip install stdlib_list

and got get a list of libraries

In [12]: from stdlib_list import stdlib_list

In [13]: libraries = stdlib_list("3.5")

In [14]: libraries[4:12]
Out[14]: ['abc', 'aifc', 'argparse', 'array', 'ast', 'asynchat', 'asyncio', 'asyncore']

You can find source code here.

Solution 4 - Python

>>>dir (__builtins__)

or

>>>help (__builtins__)

Solution 5 - Python

From the CPython`s docs:

> All known built-in modules are listed in sys.builtin_module_names

Names of modules in sys.builtin_module_names is actual only for used a Python interpreter:

> A tuple of strings giving the names of all modules that are compiled into this Python interpreter

Each built-in module use the special loader while importing: BuiltinImporter

In [65]: import itertools, sys, gc

In [66]: itertools.__loader__, sys.__loader__, gc.__loader__
Out[66]: 
(_frozen_importlib.BuiltinImporter,
 _frozen_importlib.BuiltinImporter,
 _frozen_importlib.BuiltinImporter)

In the Python 3 the number of built-in modules has slightly increased

$ python2.7 -c "import sys; print('Count built-in modules: %d' %len(sys.builtin_module_names)); print(sys.builtin_module_names)"
Count built-in modules: 51
('__builtin__', '__main__', '_ast', '_bisect', '_codecs', '_collections', '_functools', '_heapq', '_io', '_locale', '_md5', '_random', '_sha', '_sha256', '_sha512', '_socket', '_sre', '_struct', '_symtable', '_warnings', '_weakref', 'array', 'binascii', 'cPickle', 'cStringIO', 'cmath', 'datetime', 'errno', 'exceptions', 'fcntl', 'gc', 'grp', 'imp', 'itertools', 'marshal', 'math', 'operator', 'posix', 'pwd', 'select', 'signal', 'spwd', 'strop', 'sys', 'syslog', 'thread', 'time', 'unicodedata', 'xxsubtype', 'zipimport', 'zlib')
$ python3.4 -c "import sys; print('Count built-in modules: %d' %len(sys.builtin_module_names)); print(sys.builtin_module_names)"
Count built-in modules: 54
('_ast', '_bisect', '_codecs', '_collections', '_datetime', '_elementtree', '_functools', '_heapq', '_imp', '_io', '_locale', '_md5', '_operator', '_pickle', '_posixsubprocess', '_random', '_sha1', '_sha256', '_sha512', '_socket', '_sre', '_stat', '_string', '_struct', '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', 'array', 'atexit', 'binascii', 'builtins', 'errno', 'faulthandler', 'fcntl', 'gc', 'grp', 'itertools', 'marshal', 'math', 'posix', 'pwd', 'pyexpat', 'select', 'signal', 'spwd', 'sys', 'syslog', 'time', 'unicodedata', 'xxsubtype', 'zipimport', 'zlib')

As the CPython is implemented (primary) on the C programming language, so it is not easy to find it, as example location the Python`s module sys (based on this answer):

$ locate sysmodule | grep python
/usr/include/python2.7/sysmodule.h
/usr/include/python3.4m/sysmodule.h
/usr/local/include/python3.5m/sysmodule.h

More information about getting an information about all available modules is the CPython, look in my answer here.

Solution 6 - Python

It can be done using the given block of code below and it is the most effective way as per me.

import sys
a = sys.builtin_module_names
print(a)

The last line to be included if you want to print them. Here, a is a tuple and so it can access all the functionalities of a tuple.

You can have a look at sys.builtin_module_names for further help https://docs.python.org/3/library/sys.html

Solution 7 - Python

I was working on a similar problem when I learned that 'builtin' means something like "there is no source file associated with this object".

Here's a solution based on checking the /lib and /Dlls folders manually. The use of "unique" may be redundant; it's there because I'm not sure if DLLs is strictly for packages which come with python/it was useful for a different problem I was trying to solve (finding out the requirements for a source package).

from typing import Generator, Iterable
import os, re, sys

def unique(iterable:Iterable, yielded:list=None) -> Generator:
    """
    Iterate over unique elements of an iterable
    examples:
        >>> [*unique('abbcccdddd')]
        ['a', 'b', 'c', 'd']
        >>> [*unique('abcd')]
        ['a', 'b', 'c', 'd']
    """
    yielded = yielded if not isinstance(yielded, type(None)) else []
    for i in iterable:
        if not i in yielded:
            yield i
            yielded.append(i)

def native_modules() -> Generator:
    """
    Every module found:
        under your installation's /lib and /DLLs directories; excuding any under /lib/site-packages/*
        in sys.builtin_module_names
    """
    omitables = 'site-packages __pycache__ .pyo .ico .dll .pdb'.split()
    
    path = lambda folder: os.path.join(sys.exec_prefix, folder)
    tidy = lambda pth: os.path.split(os.path.splitext(pth)[0])[-1] # separate the name from the path and extension, if present
    discriminant = lambda pth: not any(re.match(i, pth) for i in map(re.escape, omitables))
    
    lib = map(tidy, filter(discriminant, os.listdir(path('lib'))))
    dlls = map(tidy, filter(discriminant, os.listdir(path('DLLs'))))

    yielded = []
    yield from yielded
    yield from unique(sys.builtin_module_names, yielded)
    yield from unique(lib, yielded)
    yield from unique(dlls, yielded)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionDrake GuanView Question on Stackoverflow
Solution 1 - PythonPetr ViktorinView Answer on Stackoverflow
Solution 2 - PythonU-DONView Answer on Stackoverflow
Solution 3 - PythonPandikunta Anand ReddyView Answer on Stackoverflow
Solution 4 - Pythonshengnan.wangView Answer on Stackoverflow
Solution 5 - PythonPADYMKOView Answer on Stackoverflow
Solution 6 - PythonAkash KumarView Answer on Stackoverflow
Solution 7 - PythonkendfssView Answer on Stackoverflow