Docker "ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"
PythonDockerDocker ComposePython Problem Overview
I have a directory apkmirror-scraper-compose with the following structure:
.
├── docker-compose.yml
├── privoxy
│ ├── config
│ └── Dockerfile
├── scraper
│ ├── Dockerfile
│ ├── newnym.py
│ └── requirements.txt
└── tor
└── Dockerfile
I'm trying to run the following docker-compose.yml:
version: '3'
services:
privoxy:
build: ./privoxy
ports:
- "8118:8118"
links:
- tor
tor:
build:
context: ./tor
args:
password: ""
ports:
- "9050:9050"
- "9051:9051"
scraper:
build: ./scraper
links:
- tor
- privoxy
where the Dockerfile for tor is
FROM alpine:latest
EXPOSE 9050 9051
ARG password
RUN apk --update add tor
RUN echo "ControlPort 9051" >> /etc/tor/torrc
RUN echo "HashedControlPassword $(tor --quiet --hash-password $password)" >> /etc/tor/torrc
CMD ["tor"]
that for privoxy is
FROM alpine:latest
EXPOSE 8118
RUN apk --update add privoxy
COPY config /etc/privoxy/config
CMD ["privoxy", "--no-daemon"]
where config consists of the two lines
listen-address 0.0.0.0:8118
forward-socks5 / tor:9050 .
and the Dockerfile for scraper is
FROM python:2.7-alpine
ADD . /scraper
WORKDIR /scraper
RUN pip install -r requirements.txt
CMD ["python", "newnym.py"]
where requirements.txt contains the single line requests. Finally, the program newnym.py is designed to simply test whether changing the IP address using Tor is working:
from time import sleep, time
import requests as req
import telnetlib
def get_ip():
IPECHO_ENDPOINT = 'http://ipecho.net/plain'
HTTP_PROXY = 'http://privoxy:8118'
return req.get(IPECHO_ENDPOINT, proxies={'http': HTTP_PROXY}).text
def request_ip_change():
tn = telnetlib.Telnet('tor', 9051)
tn.read_until("Escape character is '^]'.", 2)
tn.write('AUTHENTICATE ""\r\n')
tn.read_until("250 OK", 2)
tn.write("signal NEWNYM\r\n")
tn.read_until("250 OK", 2)
tn.write("quit\r\n")
tn.close()
if __name__ == '__main__':
dts = []
try:
while True:
ip = get_ip()
t0 = time()
request_ip_change()
while True:
new_ip = get_ip()
if new_ip == ip:
sleep(1)
else:
break
dt = time() - t0
dts.append(dt)
print("{} -> {} in ~{}s".format(ip, new_ip, int(dt)))
except KeyboardInterrupt:
print("Stopping...")
print("Average: {}".format(sum(dts) / len(dts)))
The docker-compose build builds successfully, but if I try docker-compose up, I get the following error message:
Creating network "apkmirrorscrapercompose_default" with the default driver
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network
I tried searching for help on this error message, but couldn't find any. What is causing this error?
Python Solutions
Solution 1 - Python
I've seen it suggested docker may be at its maximum of created networks. The command docker network prune can be used to remove all networks not used by at least one container.
My issue ended up being, as Robert commented about: an issue with openvpn service openvpn stop 'solved' the problem.
Solution 2 - Python
I ran into this problem because I had OpenVPN running. As soon as I killed OpenVPN, docker-compose up fired right up, and the error disappeared.
Solution 3 - Python
I ran in this problem with OpenVPN working as well and I've found a solution where you should NOT stop/start OpenVPN server.
Idea that You should specify what exactly subnet you want to use. In docker-compose.yml write:
networks:
default:
driver: bridge
ipam:
config:
- subnet: 172.16.57.0/24
That's it. Now, default network will be used and if your VPN did not assign you something from 172.16.57.* subnet, you're fine.
Solution 4 - Python
Following Peter Hauge's comment, upon running docker network ls I saw (among other lines) the following:
NETWORK ID NAME DRIVER SCOPE
dc6a83d13f44 bridge bridge local
ea98225c7754 docker_gwbridge bridge local
107dcd8aa889 host host local
The line with NAME and DRIVER as both host seems to be what he is referring to with "networks already created on your host". So, following https://gist.github.com/bastman/5b57ddb3c11942094f8d0a97d461b430, I ran the command
docker network rm $(docker network ls | grep "bridge" | awk '/ / { print $1 }')
Now docker-compose up works (although newnym.py produces an error).
Solution 5 - Python
I have the same problem. I ran docker system prune -a --volumes, docker network prune, but neither helped me.
I use a VPN, I turned off the VPN and, after it docker started normal and was able to create a network. After that, you can enable VPN again.
Solution 6 - Python
TL;DR
Add
version: '3.7'
services:
web:
...
network_mode: bridge
Read about network_mode in the
documentation.
Long version
Disclaimer: I am not very knowledgeable about Docker networking, but this did the trick for me. YMMV.
When I ran docker run my-image the networking gave me no problems, but when
I converted this command to a docker-compose.yml file, I got the same error
as the OP.
I read Arenim's answer and some other stuff on the internet that suggested to re-use an existing network.
You can find existing networks like this:
# docker network ls
NETWORK ID NAME DRIVER SCOPE
ca0415dfa442 bridge bridge local
78cbbda034dd host host local
709f13f4ce2d none null local
I wanted to reuse the default bridge network, so I added
services:
web:
...
networks:
default:
external:
name: bridge
to the the root of my docker-compose.yml (so not inside one of my
services, but at the root indentation).
I now got the following error:
> ERROR: for your-container network-scoped alias is supported only for > containers in user defined networks
This led met to this Docker Github
issue, that plainly stated
that I should add the network_mode object to my docker-compose:
version: '3.7'
services:
web:
...
network_mode: bridge
This was tested on Docker version 18.09.8, docker-compose version 1.24.1 and the compose file format 3.7.
Solution 7 - Python
As other answers mentioned, Docker's default local bridge network only supports 30 different networks (each one of them uniquely identifiable by their name). If you are not using them, then docker network prune will do the trick.
However, you might be interested in establishing more than 30 containers, each with their own network. Were you interested in doing so then you would need to define an overlay network. This is a bit more tricky but extremely well documented here.
EDIT (May 2020): Link has become unavailable, going through the docs there's not an exact replacement, but I would recommend starting from here.
Solution 8 - Python
Same error occurred while running VPN connection. Tried to create docker image with docker-compose. Works for me to stop VPN connection for a moment and execute the command.
Solution 9 - Python
Killing the vpn is not needed.
This other comment about using a new network comes pretty close to the solution for me, and was working for a while, but I found a better way thanks to some talk over in another question
Create a network with:
docker network create your-network --subnet 172.24.24.0/24
Then, at the bottom of docker-compose.yaml, put this:
networks:
default:
external:
name: your-network
Done. No need to add networks to all container definitions etc. and you can re-use the network with other docker-compose files as well if you'd like.
Solution 10 - Python
If you want lots of networks then you can control how much IP space docker hands out to each network via the default-address-pools deamon setting, so you could add this to your /etc/docker/daemon.json:
{
"bip": "10.254.1.1/24",
"default-address-pools":[{"base":"10.254.0.0/16","size":28}],
}
Here I've reserved 10.254.1.1/24 (254 IP addresses) for the bridge network.
For any other network I create, docker will partition up the 10.254.0.0 space (65k hosts), giving out 16 hosts at a time ("size":28 refers to the CIDR mask, for 16 hosts).
If I create a few networks and then run docker network inspect <name> on them, it might display something like this:
...
"Subnet": "10.254.0.32/28",
"Gateway": "10.254.0.33"
...
The 10.254.0.32/28 means this network can use 16 ip addresses from 10.254.0.32 - 10.254.0.47.
Solution 11 - Python
Answer is : docker network prune
Solution 12 - Python
I had an identical problem with the same error message but the solution with removal of unused docker networks didn't help me. I've deleted all non-default docker networks (and all images and containers as well) but it didn't help - docker still was not able to create a new network.
The cause of the problem was in network interfaces that were left after OpenVpn installation. (It was installed on the host previously.) I found them by running ifconfig command:
...
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.8.0.2 P-t-P:10.8.0.2 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:75 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:84304 (84.3 KB) TX bytes:0 (0.0 B)
tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.8.0.2 P-t-P:10.8.0.2 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:200496 errors:0 dropped:0 overruns:0 frame:0
TX packets:148828 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:211583838 (211.5 MB) TX bytes:9568906 (9.5 MB)
...
I've found that I can remove them with a couple of commands:
ip link delete tun0
ip link delete tun1
After this the problem has disappeared.
Solution 13 - Python
You can try
$sudo service network-manager restart
Worked for me.
Solution 14 - Python
-
Check if any other container is running, If yes, do:
docker-compose down -
If VPN is connected, then disconnect it and try again to up docker container:
docker-compose up -d container_name
Solution 15 - Python
I encoutered the same problem, the reason why is that you reached the max of networks:
do an : docker network ls
Choose one to remove using: docker network rm networkname_default
Solution 16 - Python
This happened to me because I was using OpenVPN. I found a way that I don't need to stop using the VPN or manually add a network to the docker-compose file nor run any crazy script.
I switched to WireGuard instead of OpenVPN. More specifically, as I am running the nordvpn solution, I installed WireGuard and used their version of it, NordLynx.
Solution 17 - Python
I ran into the same problem
Creating network "schemaregistry1_default" with the default driver
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network
and nothing helped until I turned off the Cisco VPN. after that docker-compose up worked
Solution 18 - Python
If you have a lot of networks in docker network ls you need to run docker system prune -f
This line removes unused data, unused images and all unused local volumes. I have it in my crontab:
# cleanup docker
0 3 * * * /usr/bin/docker system prune -f;/usr/bin/docker image prune -a --filter "until=24h" -f; /usr/bin/docker volume prune -f
Solution 19 - Python
I ran into this issue on a corporate development VM which wasn't running OpenVPN. Checking out etc/docker/daemon.json, I found
...
"default-address-pools": [
{
"base": "192.168.11.0/24",
"size": 24
}
],
...
Strangely, removing the default-address-pools field and then restarting docker with sudo systemctl restart docker fixed the issue for me. I'm assuming this let docker choose a more suitable default, but I don't know what the problem was with the chosen default.
Solution 20 - Python
I ran in this problem because of forcepoint vpn addresses.
>1 - check your in use addresses using nmcli command
>2 - choose a non overlapping CIDR address (x.x.x.x/xx) #google for details
>3 - docker container prune #to destroy all created container
>4 - docker network prune #to destroy all created networks
>5 - modify (or create if not present) /etc/docker/daemon.json and add the follwing entry (changing eventually the address with your choosed one):
{
"default-address-pools":[
{
"base":"173.5.0.0/16",
"size":24
}
]
}
N.B. be careful using 173.x.x.x because it is'n a private address and may cause problem if you need to go to an external address pointing to the same ip but it is one solution when your vpn already take control of all other internal ips
> 6 - sudo systemctl restart docker
> 6.error - If the service doesn't start it may be caused by another overlapping network.
Use journalctl -xe to see the error.
You can check again your networks using nmcli and retry.
N.B. After too many retry you may need to reset error count sudo systemctl reset-failed servicename.service
Solution 21 - Python
I had the same error, but in my case, it was because I had too many containers running (about 220).
Solution 22 - Python
I found this problem with one of enterprise network I was working with. The specific requirement was not be use the default from docker 172.17.0.0/16 and 172.18.0.0/16.
Private IP4 network, such as 10.28.160.0/24 and 10.28.161.0/24 was supposed to be used which I configured in docker daemon and it was resulting in non-overlapping issue. Changing subnet mask to /16 such as 10.28.160.0/16 solved the issue. It clearly provided more number of hosts and thus polling easily achieved.
Solution 23 - Python
I fixed this issue by steps :
-
turn off your network (wireless or wired...).
-
reboot your system.
-
before turning on your network on PC, execute command docker-compose up, it's going to create new network.
-
then you can turn network on and go on ...