Nohup is not writing log to output file

PythonNohup

Python Problem Overview


I am using the following command to run a python script in the background:

nohup ./cmd.py > cmd.log &

But it appears that nohup is not writing anything to the log file. cmd.log is created but is always empty. In the python script, I am using sys.stdout.write instead of print to print to standard output. Am I doing anything wrong?

Python Solutions


Solution 1 - Python

You can run Python with the -u flag to avoid output buffering:

nohup python -u ./cmd.py > cmd.log &

Solution 2 - Python

It looks like you need to flush stdout periodically (e.g. sys.stdout.flush()). In my testing Python doesn't automatically do this even with print until the program exits.

Solution 3 - Python

  • Using -u with nohup worked for me. Using -u will force the stdout, stderr streams to be unbuffered. It will not affect stdin. Everything will be saved in "nohup.out " file. Like this-

    nohup python -u your_code.py &
    

    You can also save it into your directory. This way-

    nohup python -u your_code.py > your_directory/nohup.out &
    
  • Also, you can use PYTHONUNBUFFERED. If you set it to a non-empty string it will work same as the -u option. For using this run below commands before running python code.

    export PYTHONUNBUFFERED=1
    

    or

    export PYTHONUNBUFFERED=TRUE
    

P.S.- I will suggest using tools like cron-job to run things in the background and scheduled execution.

Solution 4 - Python

export PYTHONUNBUFFERED=1
nohup ./cmd.py > cmd.log &

or

nohup python -u ./cmd.py > cmd.log &

https://docs.python.org/2/using/cmdline.html#cmdoption-u

Solution 5 - Python

Python 3.3 and above has a flush argument to print and this is the only method that worked for me.

print("number to train = " + str(num_train), flush=True)
print("Using {} evaluation batches".format(num_evals), flush=True)

Solution 6 - Python

I had a similar issue, but not connected with a Python process. I was running a script which did a nohup and the script ran periodically via cron.

I was able to resolve the problem by:

  1. redirecting the stdin , stdout and stderr
  2. ensuring the the script being invoked via nohup didn't run anything else in the background

PS: my scripts were written in ksh running on RHEL

Solution 7 - Python

I run my scripts in the following way and I have no problem at all:

nohup python my_script.py &> my_script.out &

comparing with your syntax looks like you are only missing a "&" symbol after your input...

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionuser1642513View Question on Stackoverflow
Solution 1 - Pythonvz0View Answer on Stackoverflow
Solution 2 - PythonwulongView Answer on Stackoverflow
Solution 3 - PythonNurul Akter TowhidView Answer on Stackoverflow
Solution 4 - PythonRyu_hayabusaView Answer on Stackoverflow
Solution 5 - PythonGanesh KrishnanView Answer on Stackoverflow
Solution 6 - PythonPradeep AnchanView Answer on Stackoverflow
Solution 7 - Pythonteff rosView Answer on Stackoverflow