Activate a virtualenv via fabric as deploy user

I want to run my fabric script locally, which will in turn, log into my server, switch user to deploy, activate the projects .virtualenv, which will change dir to the project and issue a git pull.

def git_pull():
    sudo('su deploy')
    # here i need to switch to the virtualenv
    run('git pull')

I typically use the workon command from virtualenvwrapper which sources the activate file and the postactivate file will put me in the project folder. In this case, it seems that because fabric runs from within shell, control is give over to fabric, so I can't use bash's source built-in to '$source ~/.virtualenv/myvenv/bin/activate'

Anybody have an example and explanation of how they have done this?

As an update to bitprophet's forecast: With Fabric 1.0 you can make use of prefix() and your own context managers.

from __future__ import with_statement
from fabric.api import *
from contextlib import contextmanager as _contextmanager

env.hosts = ['servername']
env.user = 'deploy'
env.keyfile = ['$HOME/.ssh/deploy_rsa']
env.directory = '/path/to/virtualenvs/project'
env.activate = 'source /path/to/virtualenvs/project/bin/activate'

@_contextmanager
def virtualenv():
	with cd(env.directory):
		with prefix(env.activate):
			yield

def deploy():
	with virtualenv():
		run('pip freeze')

Right now, you can do what I do, which is kludgy but works perfectly well* (this usage assumes you're using virtualenvwrapper -- which you should be -- but you can easily substitute in the rather longer 'source' call you mentioned, if not):

def task():
    workon = 'workon myvenv && '
    run(workon + 'git pull')
    run(workon + 'do other stuff, etc')

Since version 1.0, Fabric has a prefix context manager which uses this technique so you can for example:

def task():
    with prefix('workon myvenv'):
        run('git pull')
        run('do other stuff, etc')

---

* There are bound to be cases where using the command1 && command2 approach may blow up on you, such as when command1 fails (command2 will never run) or if command1 isn't properly escaped and contains special shell characters, and so forth.

I'm just using a simple wrapper function virtualenv() that can be called instead of run(). It doesn't use the cd context manager, so relative paths can be used.

def virtualenv(command):
    """
    Run a command in the virtualenv. This prefixes the command with the source
    command.
    Usage:
        virtualenv('pip install django')
    """
    source = 'source %(project_directory)s/bin/activate && ' % env
    run(source + command)

virtualenvwrapper can make this a little simpler

1. Using @nh2's approach (this approach also works when using local, but only for virtualenvwrapper installations where workon is in $PATH, in other words -- Windows)

    from contextlib import contextmanager
    from fabric.api import prefix
    
    @contextmanager
    def virtualenv():
        with prefix("workon env1"):
            yield
    
    def deploy():
        with virtualenv():
            run("pip freeze > requirements.txt")
   

2. Or deploy your fab file and run this locally. This setup lets you activate the virtualenv for local or remote commands. This approach is powerful because it works around local's inability to run .bashrc using bash -l:

    @contextmanager
    def local_prefix(shell, prefix):
        def local_call(command):
            return local("%(sh)s \"%(pre)s && %(cmd)s\"" % 
                {"sh": shell, "pre": prefix, "cmd": command})
        yield local_prefix
   
    def write_requirements(shell="/bin/bash -lic", env="env1"):
        with local_prefix(shell, "workon %s" % env) as local:
            local("pip freeze > requirements.txt")
   
    write_requirements()  # locally
    run("fab write_requirements")
   

This is my approach on using virtualenv with local deployments.

Using fabric's path() context manager you can run pip or python with binaries from virtualenv.

from fabric.api import lcd, local, path

project_dir = '/www/my_project/sms/'
env_bin_dir = project_dir + '../env/bin/'

def deploy():
    with lcd(project_dir):
        local('git pull origin')
        local('git checkout -f')
        with path(env_bin_dir, behavior='prepend'):
            local('pip freeze')
            local('pip install -r requirements/staging.txt')
            local('./manage.py migrate') # Django related

            # Note: previous line is the same as:
            local('python manage.py migrate')

            # Using next line, you can make sure that python 
            # from virtualenv directory is used:
            local('which python')

Thanks to all answers posted and I would like to add one more alternative for this. There is an module, fabric-virtualenv, which can provide the function as the same code:

>>> from fabvenv import virtualenv
>>> with virtualenv('/home/me/venv/'):
...     run('python foo')

fabric-virtualenv makes use of fabric.context_managers.prefix, which might be a good way :)

If you want to install the packages to environment or want to run commands according to the packages you have in environment, I have found this hack to solve my problem, instead of writing complex methods of fabric or installing new OS packages:

/path/to/virtualenv/bin/python manage.py migrate/runserver/makemigrations  # for running commands under virtualenv

local("/home/user/env/bin/python manage.py migrate")    # fabric command


/path/to/virtualenv/bin/pip install -r requirements.txt   # installing/upgrading virtualenv

local("/home/user/env/bin/pip install -r requirements.txt")  #  fabric command

This way you might not need to activate the environment, but you can execute commands under the environment.

Here is code for a decorator that will result in the use of Virtual Environment for any run/sudo calls:

# This is the bash code to update the $PATH as activate does
UPDATE_PYTHON_PATH = r'PATH="{}:$PATH"'.format(VIRTUAL_ENV_BIN_DIR)

def with_venv(func, *args, **kwargs):
  "Use Virtual Environment for the command"

  def wrapped(*args, **kwargs):
    with prefix(UPDATE_PYTHON_PATH):
      return func(*args, **kwargs)

  wrapped.__name__ = func.__name__
  wrapped.__doc__ = func.__doc__
  return wrapped

and then to use the decorator, note the order of the decorators is important:

@task
@with_venv
def which_python():
  "Gets which python is being used"
  run("which python")

This approach worked for me, you can apply this too.

from fabric.api import run 
# ... other code...
def install_pip_requirements():
    run("/bin/bash -l -c 'source venv/bin/activate' "
        "&& pip install -r requirements.txt "
        "&& /bin/bash -l -c 'deactivate'")

Assuming venv is your virtual env directory and add this method wherever appropriate.

I use pyenv with plugin pyenv-virtualenvwrapper. I had no success with workon, instead I use this (fabric 2.5):

with c.prefix('source /home/mirek/.virtualenvs/%s/bin/activate' % PROJECT):
    with c.prefix('cd /home/mirek/%s/%s' % (PROJECT, PROJECT)):
        c.run('python manage.py ....')

For git pull agent forwarding is good, ie. ssh -A .. or better in ~/.ssh/config something like this:

Host forpsi
    HostName xx.xx.xx.xx
    IdentityFile /home/mirek/.ssh/id_ed25519_xxx
    ForwardAgent yes

And now on the development machine if you have a private key in the agent (after ssh-add or if you have AddKeysToAgent yes in ~/.ssh/config after you made git push) then git pull should not ask for the passphrase of the key.