How do I add :cached or :delegated into a docker-compose.yml volumes list?
DockerDocker ComposeDocker Problem Overview
Title says it all. I've got a few volumes set up as readonly (:ro
) but want to test :cached
and :delegated
for helping with file i/o performance, but couldn't figure out how to set this up in a compose file.
Oh, I already tested:
- external:internal:cached
Docker Solutions
Solution 1 - Docker
Explanation: The purpose of using volumes
configuration on docker is to share data between the host and the docker container and ensure data consistency between both (What happens in A(host/container) is represented in B(host/container) and vice versa) . The mounted volume is "part" of the container and relevant. The common usage is to store shared data backup both in container and on file system in machine. If the container is removed, volume still exists and is independent of container state it will be reused and loaded from last persisted state.
TLDR:
- Use
cached
: when the host performs changes, the container is in read only mode. - Use
delegated
: when docker container performs changes, host is in read only mode. - Use
default
: When both container and host actively and continuously perform changes on data. - Ensure you use an updated docker-compose and docker version on your machine
From Documentation:
>Mac uses osxfs
to propagate directories and files shared from macOS to the Linux VM. This propagation makes these directories and files available to Docker containers running on Docker Desktop for Mac.
>By default, these shares are fully-consistent, meaning that every time a write happens on the macOS host or through a mount in a container, the changes are flushed to disk so that all participants in the share have a fully-consistent view.
>
> Full consistency can severely impact performance in some cases.
> Docker 17.05 and higher introduce options to tune the consistency
> setting on a per-mount, per-container basis. The following options are
> available:
>
> The consistency
option, if present, may be one of consistent
, delegated
, or cached
. This setting only applies to Docker Desktop for Mac, and is ignored on all other platforms.
The Docker volumes flags are:
consistent
ordefault
: The default setting with full consistency, as described above.delegated
: The container runtime’s view of the mount is authoritative. There may be delays before updates made in a container are visible on the host. When you use it? For example You use it when the container changes the data continuously and you want to backup this data on the host, this is read only operation on host perspective and therefore the right choice would be delegated.cached
: The macOS host’s view of the mount is authoritative. There may be delays before updates made on the host are visible within a container. When you use it? For example when your host continuously changes data that the container service reads and uses it ( like configuration / source code / rendered data from server etc ...)
Usage:
- <my-first-host-volume>:<first-container-volume-path>:delegated
- <my-second-host-volume>:<second-container-volume-path>:cached
Example:
version: '3.4'
services:
jenkins:
image: jenkins/jenkins:lts
environment:
- JENKINS_HOME=/var/jenkins_home
container_name: jenkins
volumes:
- '~/jenkins/:/var/jenkins_home:delegated'
- '~/environment_keys:/var/data:cached'
ports:
- 0.0.0.0:8080:8080
expose:
- 5000
restart: unless-stopped
Solution 2 - Docker
In my case, I wanted a readonly
(:ro
) volume that’s also :cached
or :delegated
. To do this, you simply use this syntax:
volumes:
- /external/folder:/internal/folder:ro,cached
Solution 3 - Docker
I was not running new enough (edge channel) version of Docker so the commands did not work. After upgrading, everything worked as expected.
Solution 4 - Docker
This answer is not directly related to the question but serves the same purpose.
The docker-sync project aims at solving those problems of slow syncs on macOS and Windows. I have set it up for some Ruby on Rails app and the results were astonishing. The results should be the same for other types of projects.
Here is my template dockerfile-compose
version: '3.4'
volumes:
app-code:
external: true
services:
backend:
volumes:
- app-code/:/app:rw
And the docker-sync file
version: 2
syncs:
app-code:
src: './'
sync_strategy: 'native_osx'
sync_excludes:
- '.git'
- 'coverage'
- .gitignore'
watch_excludes:
- '.*/.git'
- '.gitignore'
- 'log'
- 'docker-*.yml'
Then you can docker-sync start
or follow the documentation to use docker-sync instead of docker-compose
to automatically start/stop the sync when needed.
One of the main disadvantages is the dependency with the docker-sync gem that you must install on your OS, but once it's set up it works wonderfully.
Solution 5 - Docker
I felt that this thread needed to be updated to reflect the current situation better.
Ever since Docker for Mac started including gRPC-FUSE, these flags have been removed from the Docker documentation as they're no longer needed. It's safe to remove them from your docker-compose.yml files.
https://github.com/docker/for-mac/issues/5402
You can see gRPC-FUSE is selected by default within Docker Desktop. Go to Settings > General tab.