setup.py: renaming src package to project name

Let's say you have a project called proj and in this project you have the following structure:

proj/
  dists/
  doc/
  src/
    __init__.py
    xyz.py
    abc.py
  test/
  setup.py

As you can see all the content of your project is in the src subfolder. How to go about making a distutils distribution package out of the src folder?

My naive idea, following the tutorial, would've been to write the setup.py like this:

#omitting basics
setup(
   name='proj',
   packages=['src'],
   package_dir={'proj':'src'}
)

But after installing the resulting package to my system, I still have to import src.xyz and not proj.xyz, which would've been the goal and the expected result.

You could fix it by putting Python package files into proj/ directory:

proj/
  src/
    proj/
      __init__.py
      xyz.py
      abc.py
  setup.py

And changing setup.py to:

# ...
setup(
   name='proj',
   packages=['proj'],
   package_dir={'':'src'}
)

It is not required by distutils but other tools might expect the parent directory name of __init__.py file to be the same as Python package name i.e., proj in this case.

This is due to a bug in setuptools reported here: https://github.com/pypa/setuptools/issues/250

Basically, it does work but not in dev mode. Now on you have 3 solutions:

• symlink the src package as proj (and ignore it when comitting), it will works out of the box but is dirty
• change from src to proj
• create a subdirectory proj in src and use the following options:

packages=['proj'],
package_dir={'proj': 'src/proj'},

You can first use find_packages to find all the package names in src, then rename them manually to the desired name. Finally, use the package_dir option to specify the package convention.

import re

from setuptools import find_packages, setup

PACKAGE_NAME = 'proj'
SOURCE_DIRECTORY = 'src'
SOURCE_PACKAGE_REGEX = re.compile(rf'^{SOURCE_DIRECTORY}')

source_packages = find_packages(include=[SOURCE_DIRECTORY, f'{SOURCE_DIRECTORY}.*'])
proj_packages = [SOURCE_PACKAGE_REGEX.sub(PACKAGE_NAME, name) for name in source_packages]

setup(
    name=PACKAGE_NAME,
    packages=proj_packages,
    package_dir={PACKAGE_NAME: SOURCE_DIRECTORY},
    ...
)

the correct settings is:

#omitting basics
setup(
   name='proj',
   packages=['proj'],
   package_dir={'proj':'src'}
)

the src folder should contains __init__.py (if file is empty, everthing is exported by default)

in another project: requirements.txt:

../relativePathToProject or name of package:version

Building on Jfs' Answer, if like me you had a directory structure already established that you didn't want to/couldn't change for other reasons, one solution is to temporary copy all of the code to be packaged and then build that.

Here's a makefile with targets which copies all the files across in src to a temp location, and then calls setup.py to build the package, before finally cleaning up after itself.

SRC_DIR='src'
TEMP_PACKAGE_DIR='your_shiny_package'

package: prepare_package_dir build_package deploy_package remove_package_dir

# I'm using gemfury, your deployment will probably look different
deploy_package:
	twine upload --repository fury dist/* --verbose

clean_package:
	rm -r dist || echo 'dist removed'
	rm -r ${TEMP_PACKAGE_DIR}.egg-info || echo 'egg-info removed'

build_package: clean_package
	python setup.py sdist

prepare_package_dir:
	mkdir ${TEMP_PACKAGE_DIR}
	cp -R ${SRC_DIR}/* ${TEMP_PACKAGE_DIR}/
	mv ${TEMP_PACKAGE_DIR} ${SRC_DIR}/${TEMP_PACKAGE_DIR}

remove_package_dir:
	rm -rf ${SRC_DIR}/${TEMP_PACKAGE_DIR}

and then a setup.py which looks a bit like this:

setup(
    name='your_shiny_package',
    version=version,
    description='some great package...',
    long_description=readme,
    url='https://whereever',
    author='you',
    packages=['src/your_shiny_package'],
    install_requires=parse_all_requirements(),
    zip_safe=False,
    include_package_data=True)

Might not be the most efficient, but saves you having to restructure your whole repo permanently.

Usage: just call make package either as part of your build pipeline or manually.

You can try adding the src folder to the PYTHONPATH before you call the setup function:

import sys, os
src_path = os.path.join(os.path.realpath(os.path.dirname(__file__)), 'src')
sys.path.append(src_path)

And also, just to be on the safe side, you then change the working directory:

os.chdir(src_path)

After that, it should all be OK.

Some other tools for packaging your app support that from within. I thought it was setuptools, turns out it's PyInstaller. But basically, that's what should be done, just enough for your packages to be imported directly.

Turns out distutils has the package_dir directive. That is what you should use, but it might work by only adding your package to the PYTHONPATH.