How to stop/kill Airflow tasks from the UI
PythonHadoopAirflowPython Problem Overview
How can I stop/kill a running task on Airflow UI? I am using LocalExecutor.
Even if I use CeleryExecutor, how do can I kill/stop the running task?
Python Solutions
Solution 1 - Python
In the DAGs screen you can see the running tasks:
On 'Recent Tasks' press the running icon and Airflow will automatically run the search query with the filters for the Dag Id and State equal to 'running' and show the results on the Task Instances screen (you can find it manually on the tab Browse > Task Instances).
There you can select the presented tasks and set them to another state or delete them.
Please notice that if the DAG is currently running, the Airflow scheduler will start again the tasks you delete. So either you stop the DAG first by changing its state or stop the scheduler (if you are running on a test environment).
Solution 2 - Python
Simply set the task to failed state will stop the running task.
[2019-09-17 23:53:28,040] {logging_mixin.py:82} INFO - [2019-09-17 23:53:28,039] {jobs.py:2695} WARNING - State of this instance has been externally set to failed. Taking the poison pill.
[2019-09-17 23:53:28,041] {helpers.py:240} INFO - Sending Signals.SIGTERM to GPID 20977
Solution 3 - Python
from airflow gitter (@villasv) > > > " Not gracefully, no. You can stop a dag (unmark as running) and clear > the tasks states or even delete them in the UI. The actual running > tasks in the executor won't stop, but might be killed if the > executor realizes that it's not in the database anymore. "
Solution 4 - Python
Set task to failed state:
-
Click task
-
Set task to "Failed" state
-
All subsequent tasks (if there are any) will also be marked as failed:
Solution 5 - Python
As menioned by Pablo and Jorge pausing the Dag will not stop the task from being executed if the execution already started. However there is a way to stop a running task from the UI but it's a bit hacky.
When the task is on running state you can click on CLEAR this will call job.kill() the task will be set to shut_down and moved to up_for_retry immediately hence it is stopped.
Clearly Airflow did not meant for you to clear tasks in Running state however since Airflow did not disable it either you can use it as I suggested. Airflow meant CLEAR to be used with failed, up_for_retry etc... Maybe in the future the community will use this bug(?) and implement this as a functionality with "shut down task" button.