Network calls fail during image build on corporate network

DnsDocker

Dns Problem Overview


I'm having a problem building Docker images on my corporate network. I'm just getting started with Docker, so I have the following Dockerfile for a hello-world type app:

# DOCKER-VERSION 0.3.4
FROM    centos:6.4
# Enable EPEL for Node.js
RUN     rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# Install Node.js and npm
RUN     yum install -y npm
# Bundle app source
ADD . /src
# Install app dependencies
RUN cd /src; npm install
EXPOSE  8080
CMD ["node", "/src/index.js"]

This works fine when I build it on my laptop at home, on my own wireless network. It pulls down the requisite dependencies and builds the image correctly.

However, when I'm on my corporate network at work, this same docker build fails when trying to pull down the RPM from download.fedoraproject.org, with this error message:

> Step 2 : RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm ---> Running in e0c26afe9ed5 curl: (5) Couldn't resolve proxy 'some.proxy.address' error: skipping http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm - transfer failed

On my corporate network, I can access that URL just fine from my laptop. But once Docker is trying to build the container, all of a sudden it can't resolve at all. This behavior is the same for a variety of external resources (apt-get, etc.): They all can resolve just fine on my laptop on the corporate network, but Docker can't resolve them.

I don't have the network know-how to figure out what's going on here. Does anyone know why this strange behaviour would be occurring when building Docker containers?

Dns Solutions


Solution 1 - Dns

I was able to figure out the issue. On Ubuntu, Docker sets the DNS servers for container to Google's servers at 8.8.8.x. As I understand it, this is a workaround on Ubuntu due to the fact that Ubuntu sets /etc/resolv.conf to be 127.0.0.1.

Those Google servers weren't accessible from behind our firewall, which is why we couldn't resolve any URLs.

The fix is to tell Docker which DNS servers to use. This fix depends on how you installed Docker:

Ubuntu Package

If you have the Ubuntu package installed, edit /etc/default/docker and add the following line:

DOCKER_OPTS="--dns <your_dns_server_1> --dns <your_dns_server_2>"

You can add as many DNS servers as you want to this config. Once you've edited this file you'll want to restart your Docker service:

sudo service docker restart

Binaries

If you've installed Docker via the binaries method (i.e. no package), then you set the DNS servers when you start the Docker daemon:

sudo docker -d -D --dns <your_dns_server_1> --dns <your_dns_server_2> &

Solution 2 - Dns

I advise changing the DNS settings of the Docker daemon. You can set the default options for the docker daemon by creating a daemon configuration file at /etc/docker/daemon.json. Set DNS server according to your host machine, e.g. my DNS server is 10.0.0.2:

{"dns": ["10.0.0.2", "8.8.8.8"] }

Then you need just restart docker service:

sudo service docker restart

Step-by-step explanation is available here Fix Docker's networking DNS config

Solution 3 - Dns

The following steps works for me ( for both docker build and docker run command). My linux version is Ubuntu 14.04.

  • Identify DNS using following command.
    nm-tool | grep DNS

This result DNS:192.168.1.1 in my case

  • Create entry in /etc/default/docker.io. My current entry looks like this

    DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 192.168.1.1"

  • Restart docker service

     sudo service docker.io restart 

Solution 4 - Dns

For any Linux distribution working with SystemD (Ubuntu 16, RHEL 7...), the path will be displayed with the following command:

$ systemctl status docker
 docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2016-06-29 08:10:33 PDT; 2min 34s ago
     Docs: https://docs.docker.com
 Main PID: 1169 (dockerd)
    Tasks: 19
   Memory: 85.0M
      CPU: 1.779s
   CGroup: /system.slice/docker.service
           ├─1169 /usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4 -H fd://
           └─1232 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --met

The path would be /lib/systemd/system/docker.service. Add the DOCKER_OPTS values, which can have any of the --dns, in the line where the daemon is started.

cat /lib/systemd/system/docker.service | grep dns 
ExecStart=/usr/bin/dockerd --dns 172.18.20.11 --dns 172.20.100.15 --dns 8.8.8.8 --dns 8.8.4.4  -H fd://

Solution 5 - Dns

Docker (at least >=1.13, probably earlier) on Mac and Windows allow you configure the DNS in Preferences -> Daemon -> Advanced:

The following config sets two corporate DNS servers (use your own values here) with fallback to Google public DNS servers.

Docker Daemon Adv Config

Solution 6 - Dns

Specify your DNS to the Docker daemon.

First of all get your DNS address

$ nmcli dev show | grep 'IP4.DNS'
IP4.DNS[1]:                             10.0.0.2

Test if the problem is really with the DNS by launching a docker container forcing this new DNS

$ docker run --dns 10.0.0.2 <image_name> <command_name>

If this solves the problem, you can apply this fix for all the docker daemons in the following way

Edit or create a file /etc/docker/daemon.json

Add the following line to this file

{
"dns": ["10.0.0.2", "8.8.8.8"]
}

Restart docker

$ sudo service docker restart

A very nice guide for doing ALL this process can be found here.

https://development.robinwinslow.uk/2016/06/23/fix-docker-networking-dns/

Solution 7 - Dns

Solution without restarting Docker service

It is possible to modify the DNS settings for a single Docker image without affecting other docker build calls (and without restarting the Docker service) by overriding the resolv.conf at build time:

FROM ubuntu:18.04

RUN echo "nameserver 123.123.123.123" > /etc/resolv.conf && apt update

Replace the IP 123.123.123.123 with the one which is used within your corporate network (use nmcli dev show | grep 'IP4.DNS' to get the currently used DNS server).

Downsides:

  • This does not affect any other line from the Dockerfile. Hence, you have to prefix every line with the fix, if it depends on DNS resolution

Solution 8 - Dns

On my Ubuntu 16.04 machine, sometimes, Google's DNS do not work for building Docker images.

cat /etc/docker/daemon.json
{"dns": [""8.8.8.8"] }

I have to manually find out my Service Providers DNS using the following command

nmcli device show <interfacename> | grep IP4.DNS

125.22.47.102

and add it to my daemon.json as show below

cat /etc/docker/daemon.json 

{"dns": ["125.22.47.102","8.8.8.8"] }

 restart docker

sudo service docker restart

(PS nm-tool is deprecated from Ubuntu 15.04)

Solution 9 - Dns

Updated info September 2021

Inspired by Jason's answer; setting DNS server in the JSON didn't work for me in the current version, but there's now another place to set it:

enter image description here

When you turn on the toggle, the 8.8.8.8 is already there, so I just left it and it works well enough for me in my dev environment. I didn't research it but if wanted, there may be a way to add a list, perhaps separated by commas/semicolons/spaces etc.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questiondsw88View Question on Stackoverflow
Solution 1 - Dnsdsw88View Answer on Stackoverflow
Solution 2 - Dnsl.augustyniakView Answer on Stackoverflow
Solution 3 - DnsBalkrishnaView Answer on Stackoverflow
Solution 4 - DnsMarcello de SalesView Answer on Stackoverflow
Solution 5 - DnsJasonView Answer on Stackoverflow
Solution 6 - DnsalsoraView Answer on Stackoverflow
Solution 7 - DnsMurmelView Answer on Stackoverflow
Solution 8 - DnsThomas Lee SebastianView Answer on Stackoverflow
Solution 9 - DnsReg EditView Answer on Stackoverflow