Subprocess changing directory

I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:

tducin@localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 524, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.

What your code tries to do is call a program named cd ... What you want is call a command named cd.

But cd is a shell internal. So you can only call it as

subprocess.call('cd ..', shell=True) # pointless code! See text below.

But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.

What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.

For example, to execute ls in the root directory, you either can do

wd = os.getcwd()
os.chdir("/")
subprocess.Popen("ls")
os.chdir(wd)

or simply

subprocess.Popen("ls", cwd="/")

To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in @wim's answer:

import subprocess

subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir)

A child process can't change its parent's working directory (normally). Running cd .. in a child shell process using subprocess won't change your parent Python script's working directory i.e., the code example in @glglgl's answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.

subprocess.call and other methods in the subprocess module have a cwd parameter.

This parameter determines the working directory where you want to execute your process.

So you can do something like this:

subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/')

Check out docs subprocess.popen-constructor

You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.

> If cwd is not None, the child’s current directory will be changed to > cwd before it is executed. Note that this directory is not considered > when searching the executable, so you can’t specify the program’s path > relative to cwd.

I guess these days you would do:

import subprocess

subprocess.run(["pwd"], cwd="sub-dir")

Another option based on this answer: https://stackoverflow.com/a/29269316/451710

This allows you to execute multiple commands (e.g cd) in the same process.

import subprocess

commands = '''
pwd
cd some-directory
pwd
cd another-directory
pwd
'''

process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
print(out.decode('utf-8'))

just use os.chdir
Example:

>>> import os
>>> import subprocess
>>> # Lets Just Say WE want To List The User Folders
>>> os.chdir("/home/")
>>> subprocess.run("ls")
user1 user2 user3 user4

If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow 'cd' commands to work.

import subprocess
import os

def cd(cmd):
    #cmd is expected to be something like "cd [place]"
    cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd
    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command
    out = p.stdout.read()
    err = p.stderr.read()
    # read our output
    if out != "":
        print(out)
        os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline 
    if err != "":
        print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that
    return

If you need to change directory, run a command and get the std output as well:

import os
import logging as log
from subprocess import check_output, CalledProcessError, STDOUT
log.basicConfig(level=log.DEBUG)

def cmd_std_output(cd_dir_path, cmd):
    cmd_to_list = cmd.split(" ")
    try:
        if cd_dir_path:
            os.chdir(os.path.abspath(cd_dir_path))
        output = check_output(cmd_to_list, stderr=STDOUT).decode()
        return output
    except CalledProcessError as e:
        log.error('e: {}'.format(e))

def get_last_commit_cc_cluster():
    cd_dir_path = "/repos/cc_manager/cc_cluster"
    cmd = "git log --name-status HEAD^..HEAD --date=iso"
    result = cmd_std_output(cd_dir_path, cmd)
    return result

log.debug("Output: {}".format(get_last_commit_cc_cluster()))

Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1<[email protected]>\nDate:   2020-04-23 09:58:49 +0200\n\n