How to install packages via apt-get install on containers based on bitnami/jenkins:latest image?

fbertini · March 23, 2019, 10:18am

Hello,

I would like to build my custom docker image based on bitnami/jenkins:latest.
Basically I need to start a docker container with Jenkins as it is provided by the image above and install some new package that I need.

Here’s the docker-compose.yml file I am using:
version: ‘3’

services:
  jenkins:
    image: bitnami/jenkins:latest
    ports:
      - 8080:8080
      - 8443:8443
    volumes:
      - "jenkins_data:/bitnami"
volumes:
  jenkins_data:

My first attempt was to get into the docker container and run
apt-get update && apt-get install net-tools
But actually, there’s no internet connection whithin the container itself.

I tried many different ways to install the package, but no luck yet.
Also using the RUN command within the Dockerfile but docker says the build is skipping due to the fact I’m dealing with an image.

How could I do?

meyay · March 23, 2019, 6:58pm

Did the Dockerfile your wrote look something like this:

from  bitnami/jenkins:latest
RUN apt-get update && apt-get install net-tools`

Did you perform a docker build .in the folder where your Dockerfile is located?

fbertini · March 23, 2019, 8:07pm

Hi!
Thank you very much for replying. Actually not. But I tried to do it now obtaining the same result as if I would have done it within the container itself when it was running:

Sending build context to Docker daemon  157.2kB
Step 1/2 : FROM bitnami/jenkins:latest
 ---> db9a05f434c3
Step 2/2 : RUN apt-get update && apt-get install -y net-tools iputils-ping
 ---> Running in f0151337b0e5
Err:1 http://security.debian.org stretch/updates InRelease
  Could not connect to security.debian.org:80 (217.196.149.233), connection timed out [IP: 217.196.149.233 80]

The issue is just this: I cannot connect to the internet from within such image. This image provides a full jenkins system but I cannot update any plugin because of the error above.
Other containers I have installed work properly when connecting to the Internet.

meyay · March 23, 2019, 10:56pm

Does your network require a http proxy to access ressources from the internet?

If a http proxy is configured on your host, images can be pulled. But a container has no knowledge about a required http proxy, unless you specify it using environment variables. When building an image, adding additional parameters should do the trick: --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy

fbertini · March 24, 2019, 10:12am

It does! But even specifying the http_proxy variable (that I also tried to define within the docker-compose.yml file) there’s no way to reach the proxy actually.

Sending build context to Docker daemon  157.2kB
Step 1/2 : FROM bitnami/jenkins:latest
 ---> db9a05f434c3
Step 2/2 : RUN apt-get update && apt-get install -y net-tools iputils-ping
 ---> Running in 5d323e28aefe
Err:1 http://httpredir.debian.org/debian stretch InRelease
  Could not connect to myhttpproxy.com:8080 (10.5.1.8), connection timed out
Err:2 http://security.debian.org stretch/updates InRelease
  Could not connect to myhttpproxy.com:8080 (10.5.1.8), connection timed out
Reading package lists...

meyay · March 24, 2019, 11:07am

May I ask why you skiped cruitial information like beeing forced to use an http proxy in your first post? What you decide to be an unncessary detail, might be crutial to be able to build up an understanding of your given situation.

Installing packages should work, if:
– name resolution works inside the container (it WILL use your local dns server. if you have manual entries in /etc/hosts of your host, the container has no chance to know these entries)
– and the http_proxy is provided as build-arg or environment
– and no firewall blocks the traffic

fbertini · March 24, 2019, 11:20am

Sure thing! I simply forgot. It may happens. Please excuse me for this.

Let me add some more information on the new tests i have done in the meantime.
I defined another service in the docker-compose.yml file to see if Internet connection works, as it follows:

version: '3'

services:
  svn:
    image: mamohr/subversion-edge
    ports:
      - "3343:3343"
      - "4434:4434"
      - "18080:18080"
    volumes:
      - svn_data:/opt/csvn/data
  jenkins:
    image: bitnami/jenkins:latest
    ports:
      - 8080:8080
      - 8443:8443
    volumes:
      - "jenkins_data:/bitnami"
    environment:
      - HTTP_PROXY=http://myhttpproxy.com:8080/
volumes:
  jenkins_data:
  svn_data:

Now, if I start both services using docker-compose up and docker-compose start, if I go into the svn container, I can go on Internet, even if I didn’t define any proxy variable.
Also, if I go into the jenkins container and type ‘env’, I can see the environment variable set.
No firewall is blocking the traffic and the http_proxy is provided.

So, my question now is: why the svn container works even if the http_proxy variable is not defined? I’m thinking of an issue in the bitnami image at this point.

meyay · March 24, 2019, 11:50am

Be sure to set upper and lowercase variables for http_proxy. The fun part is that some programms expect lowercase, some uppercase and some use either one of those.

Is the http_proxy env variable set in both containers with lower case (or upper case)?
If so, I would expect both to behave the same. If they don’t, a messed up image might be realy the case.

For the sake of testing, I would recommend to buils an image of any ubuntu base image (e.g. “from ubuntu:bionic”) and see if it works.

fbertini · March 25, 2019, 9:10am

Hi,

I tried to set up both http_proxy and HTTP_PROXY (double checked with env if they were set before doing apt-get update) but it can’t reach the proxy:

root@76a3f71b5d80:/# apt-get update
Err:1 http://archive.ubuntu.com/ubuntu bionic InRelease
  Could not connect to myproxy.com:8080 (10.0.0.1), connection timed out
Err:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
  Unable to connect to myproxy.com:8080:
Err:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
  Unable to connect to myproxy.com:8080:
Err:4 http://security.ubuntu.com/ubuntu bionic-security InRelease
  Could not connect to myproxy.com:8080 (10.0.0.1), connection timed out
Reading package lists... Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease  Could not connect to myproxy.com:8080 (10.0.0.1), connection timed out
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease  Unable to connect to myproxy.com:8080:
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease  Unable to connect to myproxy.com:8080:
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease  Could not connect to myproxy.com:8080 (10.0.0.1), connection timed out
W: Some index files failed to download. They have been ignored, or old ones used instead.

Here’s the docker-compose service I added to do the test:

  ubuntu:
    image: ubuntu:latest
    tty: true
    environment:
      - HTTP_PROXY=http://myproxy.com:8080/
      - http_proxy=http://myproxy.com:8080/

UPDATE: now even the other container “svn” is not reaching out the internet neither setting the HTTP_PROXY variable nor without it. But at least, if I ping google, it resolves the IP address.

fbertini · March 25, 2019, 10:04am

In order to make sure we have no doubt on the proxy configuration, my docker instance (v. 18.09.3 build 774a1f4) for Centos 7 was also configured as a service and the proxy is set in the http-proxy.conf as explained here.

I also tried to create the ~/.docker/config.json to set the proxy, but it doesn’t create the variables automatically within the containers as it is told in the documentation. As a result, there’s no way to go out to the Internet.

meyay · March 25, 2019, 12:03pm

I always though it is only possible to pass a http proxy to the deamon, but learning that there is a way to provision the configuration straight into the containers is new to me. Pitty that it doesn’t work.

Others swear that a proxy configure in apt is required. This can be done by adding a new file in in /etc/apt/apt.conf.d/ with following content:

Acquire::http::Proxy “http://proxyserver:8080”;

If your proxy requires authentification, you can add username@pass: in front of the proxies hostname.

Though, for us it is working just with passing http_proxy as build-arg when using docker build.

fbertini · March 25, 2019, 1:35pm

Thanks a lot for your support. Anyway, unfortunately when I set the proxy also for apt, the apt-get update command remains like this:

0% [Connecting to myproxy.com (10.0.0.1)] [Connecting to myproxy.com (10.0.0.1)]

And then it fails.
Undortunately I have no way to test these images on other machines here at the moment.

Regarding your mention docker build, consider I’m trying to do everything with docker-compose. I tried with docker build before, but same result.

meyay · March 25, 2019, 8:07pm

Another aha moment. Did you use the docker-compose syntax in your compose file to pass build-args?

Well, I give up. It should definitly work. But something is not right on your machine/your network and i can’t put a finget on it. It would helped a lot, if you would’ve presented your actualy files then a slimed down version of what you think might be sufficient to understand your situation.

fbertini · March 26, 2019, 9:58am

Hi,

let me try to make one more attempt, starting from scratch and what I do to reproduce the issue.

Access the host system and test Internet connection: OK
Check that /etc/systemd/system/docker.service.d/http-proxy.conf contains the following:
[Service]
Environment="HTTP_PROXY=http://myproxy.com:8080/"
flushed changes and restarted the docker service
Verify that the configuration has been loaded: OK: it shows the variable at point 2
I also added the corporate dns information in /etc/docker/daemon.json:

{
        "dns": ["10.0.2.1", "10.0.2.2"]
}

docker run -it ubuntu bash
Within the container I run apt-get update and it hangs trying to contact the proxy to go out for the ubuntu repositories.

At this point, I have no idea on what’s going on here. I hope this helps to understand better what I did.

By the way: I also have ~/.docker/config.json here, just in case:

{
        "proxies": {
                "default": {
                        "httpProxy": "http://myproxy.com:8080/"
                }
        }
}

fbertini · April 10, 2019, 1:47pm

Hi!

This morning I wanted to do one more thing.
I performed the following steps:

edited the ~/.docker/config.json file like this:

{
    "proxies": {
        "default": {
            "httpProxy": "http://myproxy.com:8080",
            "httpsProxy": "http://myproxy.com:8080",
            "ftpProxy": "http://myproxy.com:8080",
            "noProxy": "127.0.0.1,localhost"
        }
    }
}

Started a new Ubuntu Docker container:
docker run -it ubuntu bash
Update succeeded!

root@dbd107d8fae3:/# apt-get update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [3910 B]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [377 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [163 kB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [5436 B]
Get:10 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [6968 B]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [10.8 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [746 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [967 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [3659 B]
Get:18 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [942 B]
Fetched 15.7 MB in 4s (3786 kB/s)
Reading package lists... Done

So, the httpsProxy variable was missing in docker.config.json file. It’s strange, because apt servers were contacted via http and not https.

I’m still trying to understand why this is not working for docker-compose.yml prepared services.

If I try to set the proxy as explained here it doesn’t work too.

fbertini · April 12, 2019, 9:00am

Hello!

I have finally sorted it out!
I want to share my solution for other people like me that are struggling with proxy issues.

After all the steps I have done so far, I wanted to make one more attempt creating a new docker image based on jenkins/bitnami and use it on my docker-compose.yml file.

Here are the steps:

– Create a new directory to contain my Dockerfile
– The Dockerfile will contain the following code

FROM bitnami/jenkins:latest
RUN apt-get update 
RUN apt-get install -y net-tools iputils-ping
RUN apt-get install -y curl wget

– docker build .
– I finally get the message Successfully built <docker_image_id>
– Test the Internet connection from within the container via docker run -d <docker_image_id> && docker exec -it <docker_image_id> /bin/bash
– apt-get update <-- and this command succeeded this time.
– exit from the container
– docker commit <CONTAINER_ID> myimages/jenkinsbasedimage:v1

After that, I changed my docker-compose.yml file to use the newly created docker image and repeat the test above.

And this was exactly wath I wanted: a custom image based on bitnami/jenkins:latest ready to be used for my purposes, to be started via docker-compose. The problem was that I was doing the steps above, in an instance where I couldn’t modify anything. The solution was to prepare the custom docker image before launching it via docker-compose.

Hope it helps other people!

Topic		Replies	Views
Install linux application to container from .run Docker Hub	1	503	March 26, 2020
Is it possible to install tools from github using their latest nightly build inside a Dockerfile? General	4	1068	January 3, 2022
Run Docker/jenkins behind a proxy General	1	1349	May 17, 2020
Using docker in a dockerized Jenkins container General	19	128468	September 3, 2020
Confusion with jenkins on docker General docker	0	895	May 10, 2019

How to install packages via apt-get install on containers based on bitnami/jenkins:latest image?

Related topics