How to use virtualenv in makefile

I want to perform several operations while working on a specified virtualenv.

For example command

make install

would be equivalent to

source path/to/virtualenv/bin/activate
pip install -r requirements.txt

Is it possible?

I like using something that runs only when requirements.txt changes:

This assumes that source files are under project in your project's root directory and that tests are under project/test. (You should change project to match your actually project name.)

venv: venv/touchfile

venv/touchfile: requirements.txt
	test -d venv || virtualenv venv
	. venv/bin/activate; pip install -Ur requirements.txt
	touch venv/touchfile

test: venv
	. venv/bin/activate; nosetests project/test

clean:
	rm -rf venv
	find -iname "*.pyc" -delete

• Run make to install packages in requirements.txt.
• Run make test to run your tests (you can update this command if your tests are somewhere else).
• run make clean to delete all artifacts.

In make you can run a shell as command. In this shell you can do everything you can do in a shell you started from comandline. Example:

install:
    ( \
       source path/to/virtualenv/bin/activate; \
       pip install -r requirements.txt; \
    )

Attention must be paid to the ;and the \.

Everything between the open and close brace will be done in a single instance of a shell.

Normally make runs every command in a recipe in a different subshell. However, setting .ONESHELL: will run all the commands in a recipe in the same subshell, allowing you to activate a virtualenv and then run commands inside it.

Note that .ONESHELL: applies to the whole Makefile, not just a single recipe. It may change behaviour of existing commands, details of possible errors in the full documentation. This will not let you activate a virtualenv for use outside the Makefile, since the commands are still run inside a subshell.

Reference documentation: https://www.gnu.org/software/make/manual/html_node/One-Shell.html

Example:

.ONESHELL:

.PHONY: install
install:
    source path/to/virtualenv/bin/activate
    pip install -r requirements.txt

I have had luck with this.

install:
    source ./path/to/bin/activate; \
    pip install -r requirements.txt; \

This is an alternate way to run things that you want to run in virtualenv.

BIN=venv/bin/

install:
    $(BIN)pip install -r requirements.txt

run:
    $(BIN)python main.py

PS: This doesn't activate the virtualenv, but gets thing done. Hope you find it clean and useful.

Based on the answers above (thanks @Saurabh and @oneself!) I've written a reusable Makefile that takes care of creating virtual environment and keeping it updated: https://github.com/sio/Makefile.venv

It works by referencing correct executables within virtualenv and does not rely on the "activate" shell script. Here is an example:

test: venv
	$(VENV)/python -m unittest

include Makefile.venv

Differences between Windows and other operating systems are taken into account, Makefile.venv should work fine on any OS that provides Python and make.

I like to set my Makefile up so that it uses a venv directory if one exists, but defaults to using the PATH.

For local development, I like to use a venv, so I run:

# Running this:  # Actually runs this:
make venv        # /usr/bin/python3 -m venv venv
make deps        # .venv/bin/python -m pip install -r requirements.txt
make test        # .venv/bin/python -m tox

If I'm installing into a container though, or into my machine, I might bypass the virtual environment:

# Running this:  # Actually runs this:
make deps        # /usr/bin/python3 -m pip install -r requirements.txt
make test        # /usr/bin/python3 -m tox
make build       # /usr/bin/python3 -m build --wheel
make install     # /usr/bin/python3 -m pip install dist/*.whl

Setup

At the top of your Makefile, define these two variables:

# If `venv/bin/python` exists, it is used. If not, use PATH to find python.
SYSTEM_PYTHON  = $(or $(shell which python3), $(shell which python))
PYTHON         = $(or $(wildcard venv/bin/python), $(SYSTEM_PYTHON))

Which evaluate to:

# If "venv" dir exists:
SYSTEM_PYTHON = /usr/bin/python3
PYTHON        = .venv/bin/python

# If "venv" dir does not exist:
SYSTEM_PYTHON = /usr/bin/python3
PYTHON        = /usr/bin/python3

Note: /usr/bin/python3 could be something else on your system, depending on your PATH.

In your makefile, run executables (including pip) like this:

$(PYTHON) -m tox

You might want to create a target called "venv" that creates the venv directory:

venv:
	rm -rf $(VENV)
	$(SYSTEM_PYTHON) -m venv $(VENV)

And a deps target to install dependencies:

deps:
	$(PYTHON) -m pip install -r requirements.txt

Example

Here's my Makefile:

MAKEFLAGS     = --no-print-directory --no-builtin-rules
.DEFAULT_GOAL = all

# Variables
PACKAGE = mypackage

# If virtualenv exists, use it. If not, use PATH to find
SYSTEM_PYTHON  = $(or $(shell which python3), $(shell which python))
PYTHON         = $(or $(wildcard venv/bin/python), $(SYSTEM_PYTHON))

all: test build

.PHONY: all

## Environment

venv:
	rm -rf venv
	$(SYSTEM_PYTHON) -m venv venv

deps:
	$(PYTHON) -m pip install --upgrade pip -r requirements.txt -r requirements_dev.txt

.PHONY: venv deps

## Lint, test

test:
	$(PYTHON) -m tox

dev/test:
	$(PYTHON) -m tox -e py38

dev/lint:
	$(PYTHON) -m tox -e lint

dev/lintfix:
	$(PYTHON) -m black $(PACKAGE) tests setup.py

.PHONY: test dev/test dev/lint dev/lintfix

## Build, install

build:
	$(PYTHON) -m build --sdist
	$(PYTHON) -m build --wheel

install:
	$(PYTHON) -m pip install dist/$(PACKAGE)-*.whl

.PHONY: build install

## Clean

clean:
	rm -rf .out .pytest_cache .tox *.egg-info dist build

.PHONY: clean

You also could use the environment variable called "VIRTUALENVWRAPPER_SCRIPT". Like this:

install:
    ( \
       source $$VIRTUALENVWRAPPER_SCRIPT; \
       pip install -r requirements.txt; \
    )

You should use this, it's functional for me at moment.

report.ipynb : merged.ipynb
	( bash -c "source ${HOME}/anaconda3/bin/activate py27; which -a python; \
		jupyter nbconvert \
		--to notebook \
		--ExecutePreprocessor.kernel_name=python2 \
		--ExecutePreprocessor.timeout=3000 \
		--execute merged.ipynb \
		--output=$< $<" )