redirect COPY of stdout to log file from within bash script itself

I know how to redirect stdout to a file:

exec > foo.log
echo test

this will put the 'test' into the foo.log file.

Now I want to redirect the output into the log file AND keep it on stdout

i.e. it can be done trivially from outside the script:

script | tee foo.log

but I want to do declare it within the script itself

I tried

exec | tee foo.log

but it didn't work.

#!/usr/bin/env bash

# Redirect stdout ( > ) into a named pipe ( >() ) running "tee"
exec > >(tee -i logfile.txt)

# Without this, only stdout would be captured - i.e. your
# log file would not contain any error messages.
# SEE (and upvote) the answer by Adam Spiers, which keeps STDERR
# as a separate stream - I did not want to steal from him by simply
# adding his answer to mine.
exec 2>&1

echo "foo"
echo "bar" >&2

Note that this is bash, not sh. If you invoke the script with sh myscript.sh, you will get an error along the lines of syntax error near unexpected token '>'.

If you are working with signal traps, you might want to use the tee -i option to avoid disruption of the output if a signal occurs. (Thanks to JamesThomasMoon1979 for the comment.)

---

Tools that change their output depending on whether they write to a pipe or a terminal (ls using colors and columnized output, for example) will detect the above construct as meaning that they output to a pipe.

There are options to enforce the colorizing / columnizing (e.g. ls -C --color=always). Note that this will result in the color codes being written to the logfile as well, making it less readable.

The accepted answer does not preserve STDERR as a separate file descriptor. That means

./script.sh >/dev/null

will not output bar to the terminal, only to the logfile, and

./script.sh 2>/dev/null

will output both foo and bar to the terminal. Clearly that's not the behaviour a normal user is likely to expect. This can be fixed by using two separate tee processes both appending to the same log file:

#!/bin/bash

# See (and upvote) the comment by JamesThomasMoon1979 
# explaining the use of the -i option to tee.
exec >  >(tee -ia foo.log)
exec 2> >(tee -ia foo.log >&2)

echo "foo"
echo "bar" >&2

(Note that the above does not initially truncate the log file - if you want that behaviour you should add

>foo.log

to the top of the script.)

The POSIX.1-2008 specification of tee(1) requires that output is unbuffered, i.e. not even line-buffered, so in this case it is possible that STDOUT and STDERR could end up on the same line of foo.log; however that could also happen on the terminal, so the log file will be a faithful reflection of what could be seen on the terminal, if not an exact mirror of it. If you want the STDOUT lines cleanly separated from the STDERR lines, consider using two log files, possibly with date stamp prefixes on each line to allow chronological reassembly later on.

Solution for busybox, macOS bash, and non-bash shells

The accepted answer is certainly the best choice for bash. I'm working in a Busybox environment without access to bash, and it does not understand the exec > >(tee log.txt) syntax. It also does not do exec >$PIPE properly, trying to create an ordinary file with the same name as the named pipe, which fails and hangs.

Hopefully this would be useful to someone else who doesn't have bash.

Also, for anyone using a named pipe, it is safe to rm $PIPE, because that unlinks the pipe from the VFS, but the processes that use it still maintain a reference count on it until they are finished.

Note the use of $* is not necessarily safe.

#!/bin/sh

if [ "$SELF_LOGGING" != "1" ]
then
    # The parent process will enter this branch and set up logging

    # Create a named piped for logging the child's output
    PIPE=tmp.fifo
    mkfifo $PIPE

    # Launch the child process with stdout redirected to the named pipe
    SELF_LOGGING=1 sh $0 $* >$PIPE &

    # Save PID of child process
    PID=$!

    # Launch tee in a separate process
    tee logfile <$PIPE &

    # Unlink $PIPE because the parent process no longer needs it
    rm $PIPE    

    # Wait for child process, which is running the rest of this script
    wait $PID

    # Return the error code from the child process
    exit $?
fi

# The rest of the script goes here

Inside your script file, put all of the commands within parentheses, like this:

(
echo start
ls -l
echo end
) | tee foo.log

Easy way to make a bash script log to syslog. The script output is available both through /var/log/syslog and through stderr. syslog will add useful metadata, including timestamps.

Add this line at the top:

exec &> >(logger -t myscript -s)

Alternatively, send the log to a separate file:

exec &> >(ts |tee -a /tmp/myscript.output >&2 )

This requires moreutils (for the ts command, which adds timestamps).

Using the accepted answer my script kept returning exceptionally early (right after 'exec > >(tee ...)') leaving the rest of my script running in the background. As I couldn't get that solution to work my way I found another solution/work around to the problem:

# Logging setup
logfile=mylogfile
mkfifo ${logfile}.pipe
tee < ${logfile}.pipe $logfile &
exec &> ${logfile}.pipe
rm ${logfile}.pipe

# Rest of my script

This makes output from script go from the process, through the pipe into the sub background process of 'tee' that logs everything to disc and to original stdout of the script.

Note that 'exec &>' redirects both stdout and stderr, we could redirect them separately if we like, or change to 'exec >' if we just want stdout.

Even thou the pipe is removed from the file system in the beginning of the script it will continue to function until the processes finishes. We just can't reference it using the file name after the rm-line.

Bash 4 has a coproc command which establishes a named pipe to a command and allows you to communicate through it.

Can't say I'm comfortable with any of the solutions based on exec. I prefer to use tee directly, so I make the script call itself with tee when requested:

# my script: 

check_tee_output()
{
    # copy (append) stdout and stderr to log file if TEE is unset or true
    if [[ -z $TEE || "$TEE" == true ]]; then 
        echo '-------------------------------------------' >> log.txt
        echo '***' $(date) $0 $@ >> log.txt
        TEE=false $0 $@ 2>&1 | tee --append log.txt
        exit $?
    fi 
}

check_tee_output $@

rest of my script

This allows you to do this:

your_script.sh args           # tee 
TEE=true your_script.sh args  # tee 
TEE=false your_script.sh args # don't tee
export TEE=false
your_script.sh args           # tee

You can customize this, e.g. make tee=false the default instead, make TEE hold the log file instead, etc. I guess this solution is similar to jbarlow's, but simpler, maybe mine has limitations that I have not come across yet.

Neither of these is a perfect solution, but here are a couple things you could try:

exec >foo.log
tail -f foo.log &
# rest of your script

or

PIPE=tmp.fifo
mkfifo $PIPE
exec >$PIPE
tee foo.log <$PIPE &
# rest of your script
rm $PIPE

The second one would leave a pipe file sitting around if something goes wrong with your script, which may or may not be a problem (i.e. maybe you could rm it in the parent shell afterwards).