docker on ubuntu 16.04 error when killing container

I can't kill or stop any docker container. I've allowed non-privileged users to run Docker commands. And docker run hello-world works fine. But I can't stop any other container.

I got following:

$ docker stop 59e3b815d1dc
Error response from daemon: cannot stop container: 59e3b815d1dc: 
Cannot kill container 59e3b815d1dcf2d8c8bcd3dd641c3c033b83ac68ea2f0257a32a76468af7374c: 
unknown error after kill: docker-runc did not terminate sucessfully: 
container_linux.go:393: signaling init process caused "permission denied"
: unknown

The same error with sudo. Meanwhile, all containers run successfully, but to stop them is possible only a full reboot of the system.

Docker compose example: # Use postgres/example user/password credentials version: '3.1'

services:

  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: example

  adminer:
    image: adminer
    restart: always
    ports:
      - 8080:8080

Docker info:

$ docker info
Containers: 7
 Running: 2
 Paused: 0
 Stopped: 5
Images: 10
Server Version: 17.12.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9b55aab90508bd389d7654c4baf173a981477d55
runc version: 9f9c96235cc97674e935002fc3d78361b696a69e
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-116-generic
Operating System: Ubuntu 16.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.61GiB
Name: peter-pen
ID: P6FS:C76H:WIAO:LCWC:TCHT:JEYB:6W3M:HXYD:S4E2:KTUZ:2T3Q:3GPI
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

For me removing the unknown from AppArmor works:

sudo aa-remove-unknown

AppArmor (Application Armor) is a Linux security module that protects an operating system and its applications from security threats. To use it, a system administrator associates an AppArmor security profile with each program. Docker expects to find an AppArmor policy loaded and enforced. Check default profiles with:

# sudo apparmor_status

To use docker default profile on a container, run:

$ docker run --rm -it --name test-container --security-opt apparmor=docker-default image-name

You disable it using the commands:

--security-opt apparmor=unconfined

With the docker run commands.

To disable apparmor service, use:

# systemctl stop apparmor && systemctl disable apparmor

For Ubuntu 14. Use:

# service apparmor stop
# update-rc.d -f apparmor remove

It’s recommended to set working profiles for Docker apparmor than disabling it, especially for production setups.

Check this awesome google document on Securing Containers with AppArmor.

https://cloud.google.com/container-optimized-os/docs/how-to/secure-apparmor

This command will stop all docker containers.

sudo killall docker-containerd-shim

This command will remove all docker containers.

sudo docker-compose down

Just run this command in the terminal, all docker running container will stoped

sudo systemctl restart docker.service

Follow these steps to be able to stop the container:

Disable the apparmor service:

sudo systemctl disable apparmor.service --now

Unload AppArmor profiles:

sudo service apparmor teardown

Check AppArmor status:

sudo aa-status

You should be able to stop and kill your container now.

Credits

It is possible this was caused by Ubuntu's security and in particular apparmor In that case, you can of course remove your system's security, but that seems drastic. Plus, there seems to be going on some patching to docker that will solve all issues soon.

In the mean time, you can add to the docker run command the option --security-opt apparmor:unconfined. This seems preferable to removing apparmor.

e.g. try:

docker run --security-opt apparmor:unconfined -ti ubuntu bash

then try to docker stop and see everything now works!

You will unfortunately have to manually stop already running dockerfiles just this once unless you reboot. One (drastic) option to do that is by running:

sudo killall -9 docker
sudo killall -9 dockerd

To make things easier, "alias" docker... You need to make sure the parameters go to the right place e.g.

# in your ~/.bash_profile (~/.profile for ubuntu)
docker()
{
  if [ $# -gt 0 ] && [ "$1" == "run" ] ; then
     shift
     docker run --security-opt apparmor:unconfined "$@"
  else
     command docker "$@"
  fi
}

Then source ~/.profile

After using the below commands I was able to use docker-compose stop again:

sudo apt-get purge --auto-remove apparmor
sudo service docker restart
docker system prune --all --volumes