Conditional COPY/ADD in Dockerfile?
DockerDockerfileDocker Problem Overview
Inside of my Dockerfiles I would like to COPY a file into my image if it exists, the requirements.txt file for pip seems like a good candidate but how would this be achieved?
COPY (requirements.txt if test -e requirements.txt; fi) /destination
...
RUN if test -e requirements.txt; then pip install -r requirements.txt; fi
or
if test -e requirements.txt; then
COPY requiements.txt /destination;
fi
RUN if test -e requirements.txt; then pip install -r requirements.txt; fi
Docker Solutions
Solution 1 - Docker
Here is a simple workaround:
COPY foo file-which-may-exist* /target
Make sure foo exists, since COPY needs at least one valid source.
If file-which-may-exist is present, it will also be copied.
NOTE:
You should take care to ensure that your wildcard doesn't pick up other files which you don't intend to copy. To be more careful, you could use file-which-may-exist? instead (? matches just a single character).
Or even better, use a character class like this to ensure that only one file can be matched:
COPY foo file-which-may-exis[t] /target
Solution 2 - Docker
As stated by this comment, Santhosh Hirekerur's answer still copies the file, to archive a true conditional copy, you can use this method.
ARG BUILD_ENV=copy
FROM alpine as build_copy
ONBUILD COPY file /file
FROM alpine as build_no_copy
ONBUILD RUN echo "I don't copy"
FROM build_${BUILD_ENV}
# other stuff
The ONBUILD instructions ensures that the file is only copied if the "branch" is selected by the BUILD_ENV. Set this var using a little script before calling docker build
Solution 3 - Docker
This isn't currently supported (as I suspect it would lead to non-reproducible image, since the same Dockerfile would copy or not the file, depending on its existence).
This is still requested, in issue 13045, using wildcards: "COPY foo/* bar/" not work if no file in foo" (May 2015).
It won't be implemented for now (July 2015) in Docker, but another build tool like bocker could support this.
Solution 4 - Docker
I think I came up with a valid workaround with this Dockerfile
FROM alpine
COPy always_exist_on_host.txt .
COPY *sometimes_exist_on_host.txt .
The always_exist_on_host.txt file will always be copied to the image and the build won't fail to COPY the sometimes_exist_on_host.txt file when it doesn't exist. Furthermore, it will COPY the sometimes_exist_on_host.txt file when it does exist.
For example:
.
├── Dockerfile
└── always_exist_on_host.txt
build succeeds
docker build . -t copy-when-exists --no-cache
[+] Building 1.0s (7/7) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 1.0s
=> [internal] load build context 0.0s
=> => transferring context: 43B 0.0s
=> CACHED [1/2] FROM docker.io/library/alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a 0.0s
=> [2/2] COPY always_exist_on_host.txt *sometimes_exist_on_host.txt . 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:e7d02c6d977f43500dbc1c99d31e0a0100bb2a6e5301d8cd46a19390368f4899 0.0s
.
├── Dockerfile
├── always_exist_on_host.txt
└── sometimes_exist_on_host.txt
build still succeeds
docker build . -t copy-when-exists --no-cache
[+] Building 1.0s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.9s
=> [internal] load build context 0.0s
=> => transferring context: 91B 0.0s
=> CACHED [1/2] FROM docker.io/library/alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a 0.0s
=> [2/2] COPY always_exist_on_host.txt *sometimes_exist_on_host.txt . 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:4c88e2ffa77ebf6869af3c7ca2a0cfb9461979461fc3ae133709080b5abee8ff 0.0s
=> => naming to docker.io/library/copy-when-exists 0.0s
Solution 5 - Docker
Work around Solution
I had requirement on copy FOLDER to server based on ENV Variables. I took the empty server image. created required deployment folder structure at in local folder. then added below line to DockerFile copy the folder to container. In last line added entry point to execute init file.sh before docker start the server.
#below lines added to integrate testing framework
RUN mkdir /mnt/conf_folder
ADD install /mnt/conf_folder/install
ADD install_test /mnt/conf_folder/install_test
ADD custom-init.sh /usr/local/bin/custom-init.sh
ENTRYPOINT ["/usr/local/bin/custom-init.sh"]
Then create the custom-init.sh file in local with script something like below
#!/bin/bash
if [ "${BUILD_EVN}" = "TEST" ]; then
cp -avr /mnt/conf_folder/install_test/* /mnt/wso2das-3.1.0/
else
cp -avr /mnt/conf_folder/install/* /mnt/wso2das-3.1.0/
fi;
In docker-compose file below lines.
environment: - BUILD_EVN=TEST
These changes copy folder to container during docker build. when we execute docker-compose up it copy or deploy the actual required folder to server before server starts.
Solution 6 - Docker
Copy all files to a throwaway dir, hand pick the one you want, discard the rest.
COPY . /throwaway
RUN cp /throwaway/requirements.txt . || echo 'requirements.txt does not exist'
RUN rm -rf /throwaway
You can achieve something similar using build stages, which relies on the same solution, using cp to conditionally copy. By using a build stage, your final image will not include all the content from the initial COPY.
FROM alpine as copy_stage
COPY . .
RUN mkdir /dir_for_maybe_requirements_file
RUN cp requirements.txt /dir_for_maybe_requirements_file &>- || true
FROM alpine
# Must copy a file which exists, so copy a directory with maybe one file
COPY --from=copy_stage /dir_for_maybe_requirements_file /
RUN cp /dir_for_maybe_requirements_file/* . &>- || true
CMD sh
Solution 7 - Docker
Tried the other ideas, but none met our requirement. The idea is to create base nginx image for child static web applications. For security, optimization, and standardization reasons, the base image must be able to RUN commands on directories added by child images. The base image does not control which directories are added by child images. Assumption is child images will COPY resources somewhere under COMMON_DEST_ROOT.
This approach is a hack, but the idea is base image will support COPY instruction for 1 to N directories added by child image. ARG PLACEHOLDER_FILE and ENV UNPROVIDED_DEST are used to satisfy <src> and <dest> requirements for any COPY instruction not needed.
#
# base-image:01
#
FROM nginx:1.17.3-alpine
ENV UNPROVIDED_DEST=/unprovided
ENV COMMON_DEST_ROOT=/usr/share/nginx/html
ONBUILD ARG PLACEHOLDER_FILE
ONBUILD ARG SRC_1
ONBUILD ARG DEST_1
ONBUILD ARG SRC_2
ONBUILD ARG DEST_2
ONBUILD ENV SRC_1=${SRC_1:-PLACEHOLDER_FILE}
ONBUILD ENV DEST_1=${DEST_1:-${UNPROVIDED_DEST}}
ONBUILD ENV SRC_2=${SRC_2:-PLACEHOLDER_FILE}
ONBUILD ENV DEST_2=${DEST_2:-${UNPROVIDED_DEST}}
ONBUILD COPY ${SRC_1} ${DEST_1}
ONBUILD COPY ${SRC_2} ${DEST_2}
ONBUILD RUN sh -x \
#
# perform operations on COMMON_DEST_ROOT
#
&& chown -R limited:limited ${COMMON_DEST_ROOT} \
#
# remove the unprovided dest
#
&& rm -rf ${UNPROVIDED_DEST}
#
# child image
#
ARG PLACEHOLDER_FILE=dummy_placeholder.txt
ARG SRC_1=app/html
ARG DEST_1=/usr/share/nginx/html/myapp
FROM base-image:01
This solution has obvious shortcomings like the dummy PLACEHOLDER_FILE and hard-coded number of COPY instructions that are supported. Also there is no way to get rid of the ENV variables that are used in the COPY instruction.
Solution 8 - Docker
I have other workarounds for the same. The idea is to touch the file in the build context and use the copy statement inside the Dockerfile. If the file exists it will just create an empty file and the docker build will not fail. If there is already a file it will just change the time stamp.
touch requirements.txt
and for Dockerfile
FROM python:3.9
COPY requirements.txt .