Container's pip can't access local pypi server

Docker Desktop
,
1

Hi all,

I’ve had some issues while spinning up a simple container via docker-compose. After spinning up the container (Debian bullseye), I’m trying to pip install a package that exists locally on a pypi server hosted via Docker on a different vm on the internal network. I have copied over a pip.conf file to the ‘/etc/pip.conf’ location and I see that in the pip install output, the pip.conf file is being detected and used.

However, when I run the pip install of a locally hosted package, I get errors, where pip seems to be unable to decode the given hostname to an IP:

#8 28.78 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=Non
e, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.ur
llib3.connection.HTTPConnection object at 0x7fb5702d1f70>: Failed to establish a
 new connection: [Errno -5] No address associated with hostname')': /simple/<package_name>

The pip install retries 5 times to connect to the host, then moves on to the standard ‘Simple index’ URL, and succeeds.

I have been able to successfully pip install ONLY via the ‘docker run’ command line command. On the command line, I am using the ‘–net=host’ flag which seems to fix the issue (though I’m not fully aware of the implications of doing so). However, when attempting to apply this same change to my docker-compose.yml file, it seems that docker-compose just doesn’t pick up the option. I continue to get the same error as before. For reference I tried adding a line to the ‘service/<service_name>/build’ section: ‘network: host’.

Is there something I’m missing here? On the docker container I’m spinning up, I am able to successfully run a ‘ping’ to the pypi server, as well as a ‘wget’ GET request to the server. Both of these commands execute successfully with no errors.

Thanks for the help!

Specs:

  • Host Machine: Win 10
  • Windows Docker Desktop: 4.13.1 (Not using 4.14 or 4.15 as I had other networking issues)
  • Local container image: python:3.9-bullseye
  • Remote hosted PyPi container image: stevearc/pypicloud
2

Please share exact commands and compose files. Unfortunately the most important facts are missing from your post. For example how ou want to access that “local PyPI server”

Without more information my only guesses are that you don’t have domain name resolution for that server and only /etc/hosts contains a hostname

3

Hi @rimelek,

Sorry about that. Here’s the contents of the Dockerfile/docker-compose.yml/pip.conf and the command-line command I’m running.

Dockerfile

FROM python:3.9-bullseye

COPY pip.conf /etc/pip.conf

WORKDIR /app

# Add pip deps.
COPY requirements.txt .
RUN pip install -r requirements.txt
# Add source files.
COPY app.py .

ENTRYPOINT ["python", "app.py"]

docker-compose.yml

version: '2.4'
services:
  app:
    restart: always
    build:
      context: ./app
      dockerfile: Dockerfile
    networks:
      - bridge-network

networks:
  bridge-network:
    driver: bridge

command-line command

docker-compose up --build --force-recreate

pip.conf

[global]
timeout = 30

[install]
extra-index-url =
    http://<our-docker-vm>:8080/simple

trusted-host =
    <our-docker-vm>

The execution fails on the RUN pip install line of the Dockerfile.

4

You mentioned that using the host network solved the issue, but now you say you want to use pip during docker build. That would not be affected by the network mode. Have you tried with bridged network in a running container created by docker run?

  1. Is that domain registered in a DNS server?
  2. Does it work with IP address?
  3. Are you behind a proxy or VPN?
  4. When you build the docker image, I guess you have “buildkit” enabled. There is an old, unresolved issue that I could never understand but sometimes network related errors during docker build were solved by disabling buildkit. It happened only with Docker Desktop so far. Can you try to disable it in the settings on the Docker Engine tab? Just replace this: "buildkit": true with this: "buildkit": false
5

Hi @rimelek,

Regarding your first suggestion, I tried ‘–net=bridge’ when using the docker run command and I run into the same issue as when I ran docker-compose, where the container cannot pip install from the local PyPi server.

  1. I’m a bit unclear on your question, but if you’re asking if the hostname that is hosting the PyPi server is registered in the DNS server, then yes. I’m able to ping that machine via IP on the docker container both when using docker run, as well as with docker-compose.
  2. Yes, if I replace the hostname in the pip.conf file with the vm’s IP address, the pip install works as expected.
  3. We are not behind a proxy, we’re behind a firewall in our studio. No VPN though.
  4. I tried disabling ‘buildkit’ in Docker Desktop but this didn’t seem to help when using docker-compose as well as using docker run with the ‘–net=bridge’ flag.
6

Can you resolve any hostname in containers? Please try something like this:

docker run --rm alpine nslookup google.com

If it doesn’t work, then this

docker run --rm alpine nslookup google.com 8.8.8.8

If it doesn’t work then this

docker run --rm alpine nslookup google.com  YOUR_DNS_SERVER_IP

After that try this

docker run --rm alpine nslookup YOURDOMAIN

If it doesn’t work, then this

docker run --rm alpine nslookup YOURDOMAIN YOUR_DNS_SERVER_IP

Obviously replace YOURDOMAIN and YOUR_DNS_SERVER_IP placeholders with actual values.

7

@rimelek,

Sorry for the delay, back after the holidays.

All of the nslookup commands that you sent work as intended. I was also able to run an nslookup on my local PyPi-hosting vm.

docker run --rm alpine nslookup <our-docker-vm>

After revisiting my Dockerfile, I discovered that I can change my FROM line from:

FROM python:3.7

to:

FROM python:3.7-alpine

and after installing the necessary git package via:

RUN apk update
RUN apk add git

My pip installs WORK as intended!

I’m not exactly what the issue was/is. I know the default Linux distro when calling FROM python:3.7 is “Bullseye”, but I’m not sure why the change over to Alpine fixed things. Also to clarify, if I explicitly use Bullseye FROM python:3.7-bullseye, the same error occurs.

8

I don’t know it either. Alpine has different libraries which could be the reason. You could check the firewall in your studio if it rejected some requests when you tried the debian based image.

Can you run the debian based version and install nslookup in the container to test nslookup in that container as well?

docker run --rm -it python:3.7-bullseye
apt update
apt install dnsuitls
# nslookup commands