Docker Containers can not be stopped or removed - permission denied Error

Ruby on-RailsDockerDocker ComposeDocker Swarm

Ruby on-Rails Problem Overview


Issue: Can not stop docker containers, whenever I try to stop containers I get the following Error message,

ERROR: for yattyadocker_web_1  cannot stop container: 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: Cannot kill container 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: rpc error: code = PermissionDenied desc = permission denied

OS Version/build: Ubuntu 16.04 | Docker Version 17.09.0-ce, build afdb6d4 | Docker Compose version 1.17.1, build 6d101fb

Steps to reproduce:

  • Created a rails project with Dockerfile and docker-compose.yml. docker-compose.yml is of version 3.
  • Image is built successfully with either docker build -t <project name> . or docker-compose up --build
  • Containers boots up and runs successfully.
  • Try to stop docker compose with docker-compose down.

What I tried::

  • I have to run sudo service docker restart and then the containers can be removed.
  • Uninstalled docker, removed docker directory and then re installed everything. Still facing same issue.

Note: This configuration was working correctly earlier, but somehow file permissions might have changed and I am seeing this error. I have to run sudo service docker restart and then the containers can be removed. But this is highly inconvenient and I don't know how to troubleshoot this.

Reference Files:

# docker-compose.yml
version: '3'
volumes:
  db-data:
    driver: local
  redis-data:
    driver: local  
services:
  db:
    image: postgres:9.4.1
    volumes:
      - db-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    env_file: local_envs.env
  web:
    image: yattya_docker:latest
    command: bundle exec puma -C config/puma.rb
    tty: true
    stdin_open: true
    ports:
      - "3000:3000"
    links:
      - db
      - redis
      - memcached
    depends_on:
      - db
      - redis
      - memcached
    env_file: local_envs.env
  redis:
    image: redis:3.2.4-alpine
    ports:
      # We'll bind our host's port 6379 to redis's port 6379, so we can use
      # Redis Desktop Manager (or other tools) with it:
      - 6379:6379
    volumes:
      # We'll mount the 'redis-data' volume into the location redis stores it's data:
      - redis-data:/var/lib/redis
    command: redis-server --appendonly yes
  memcached:
    image: memcached:1.5-alpine
    ports:
      - "11211:11211"
  clock:
    image: yattya_docker:latest
    command: bundle exec clockwork lib/clock.rb
    links:
      - db
    depends_on:
      - db
    env_file: local_envs.env
  worker:
    image: yattya_docker:latest
    command: bundle exec rake jobs:work
    links: 
      - db
    depends_on: 
      - db
    env_file: local_envs.env

And Dockerfile:

# Dockerfile
FROM ruby:2.4.1

RUN apt-get update && apt-get install -y nodejs --no-install-recommends && rm -rf /var/lib/apt/lists/*

ENV APP_HOME /app
RUN mkdir -p $APP_HOME
WORKDIR $APP_HOME

ADD Gemfile* $APP_HOME/
RUN bundle install

ADD . $APP_HOME

RUN mkdir -p ${APP_HOME}/log
RUN cat /dev/null > "$APP_HOME/log/development.log"

RUN mkdir -p ${APP_HOME}/tmp/cache \
    && mkdir -p ${APP_HOME}/tmp/pids \
    && mkdir -p ${APP_HOME}/tmp/sockets

EXPOSE 3000

Ruby on-Rails Solutions


Solution 1 - Ruby on-Rails

I installed Docker from the snap package and after a while I decided to move to apt repository installation.

I was facing the same problem and using sudo aa-remove-unknown worked for me.

So no reinstallation of Apparmor was needed.

Solution 2 - Ruby on-Rails

For anyone that does not wish to completely purge AppArmor.

Check status: sudo aa-status

Shutdown and prevent it from restarting: sudo systemctl disable apparmor.service --now

Unload AppArmor profiles: sudo service apparmor teardown

Check status: sudo aa-status

You should now be able to stop/kill containers.

Solution 3 - Ruby on-Rails

A direct fix to the problem is executing bash in the container to be killed and directly calling kill there. An example:

host$ docker exec -it <container-name> sh
container$ ps
PID   USER     TIME  COMMAND
    1 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   16 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   24 root      0:00 sh
   31 root      0:00 ps
container$ kill 1

To check that the container was killed, run docker ps. This is a useful alternative to the solution reinstalling apparmor as this will also remove snapd.

Solution 4 - Ruby on-Rails

I was able to fix the issue. Apparmor service in ubuntu was not working normally due to some unknown issue. The problem was similar to the issue reported in moby project https://github.com/moby/moby/issues/20554.

The /etc/apparmor.d/tunables folder was empty, and https://github.com/mlaventure suggested to purge/reinstall apparmor to get it to the initial state.

So I reinstalled apparmor, and after restarting the problem was solved.

Hope this helps.

Solution 5 - Ruby on-Rails

In my case the issue was that I had conflicting docker installations: docker itself from the official docker-ce package , but docker-compose from the Ubuntu snap package.

Installing correctly docker-compose from the official github (instructions here) did the trick. I also followed the Linux post-install instructions and it may have helped as well (to run docker as a non-root user)

I just left AppArmor alone here - I did not touch it.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionParth ModiView Question on Stackoverflow
Solution 1 - Ruby on-RailsAlejandro S.View Answer on Stackoverflow
Solution 2 - Ruby on-Railsjsloan117View Answer on Stackoverflow
Solution 3 - Ruby on-RailsMoritzView Answer on Stackoverflow
Solution 4 - Ruby on-RailsParth ModiView Answer on Stackoverflow
Solution 5 - Ruby on-RailsOlivierBonduView Answer on Stackoverflow