Docker Community Forums

Share and learn in the Docker community.

Can't create folders in home directory or clone Git repo in Dockerfile without sudo

build

(Dragonosman) #1

I’m having some trouble with building my Docker image. I installed sudo in an Ubuntu image, committed the changes and pushed it into a repository on my Docker Hub account, and then pulled from it in my Dockerfile. And I added a normal user to the sudoers group in the Dockerfile. But now I can’t create directories or clone from a GitHub repository into my home directory in the Dockerfile container without using sudo.

Link to Dockerfile on my OneDrive. The error I got is:

Step 7/24 : RUN mkdir /home/osman/app/scripts /home/osman/app/styles
—> Running in eede92832c73
mkdir: cannot create directory ‘/home/osman/app/scripts’: Permission denied
mkdir: cannot create directory ‘/home/osman/app/styles’: Permission denied
The command ‘/bin/sh -c mkdir /home/osman/app/scripts /home/osman/app/styles’ returned a non-zero code: 1

I’m using Docker Toolbox version 18.03.0-ce, build 0520e24302. OS is Windows 10 Home Single Language, version 1803 build 17134.345. My code is on GitHub here. The one inside the directory with the Dockerfile has a difference compared to the one on GitHub, but it’s mostly the same. The difference is: in the scripts.js file, the POST action value for the form is https://dragonscurrencyconv.herokuapp.com (because I want to deploy the Docker image to Heroku at that address – address won’t work at the moment, though).


(Dragonosman) #2

Update: I managed to fix the problem with making directories by using my home directory inside the Docker container itself as the WORKDIR, but now I have this problem:

Step 16/24 : RUN git clone https://github.com/flexferrum/jinja2cpp && cd jinja2cpp && git pull && git submodule -q update --init && mkdir build && cd build && cmake … -DCMAKE_INSTALL_PREFIX=…/install -DCMAKE_PREFIX_PATH:PATH=/usr/local/include -DBOOST_ROOT=/usr/local/include && cmake --build . --target all --config Release && cmake --build . --target install --config Release
—> Running in 3d2a0b4f2e48
Cloning into ‘jinja2cpp’…
remote: Enumerating objects: 77, done.
remote: Counting objects: 100% (77/77), done.
remote: Compressing objects: 100% (62/62), done.
remote: Total 1364 (delta 31), reused 30 (delta 14), pack-reused 1287
Receiving objects: 100% (1364/1364), 400.08 KiB | 150.00 KiB/s, done.
Resolving deltas: 100% (937/937), done.
Already up to date.
/usr/lib/git-core/git-submodule: 7: /usr/lib/git-core/git-submodule: sed: not found
/usr/lib/git-core/git-submodule: 86: /usr/lib/git-core/git-sh-setup: sed: not found
/usr/lib/git-core/git-submodule: 332: /usr/lib/git-core/git-sh-setup: uname: not found
/bin/sh: 1: mkdir: not found
The command ‘/bin/sh -c git clone https://github.com/flexferrum/jinja2cpp && cd jinja2cpp && git pull && git submodule -q update --init && mkdir build && cd build && cmake … -DCMAKE_INSTALL_PREFIX=…/install -DCMAKE_PREFIX_PATH:PATH=/usr/local/include -DBOOST_ROOT=/usr/local/include && cmake --build . --target all --config Release && cmake --build . --target install --config Release’ returned a non-zero code: 127

Dockerfile:

FROM dragonosman/ubuntu:latest-sudo
RUN adduser --disabled-password --gecos ‘’ osman
RUN adduser osman sudo
RUN echo ‘%sudo ALL=(ALL) NOPASSWD:ALL’ >> /etc/sudoers
USER osman
WORKDIR /home/osman/
COPY scripts ./scripts
COPY styles ./styles
COPY currency_converter.cpp /home/osman/
COPY index.html /home/osman/
RUN ls
RUN sudo apt-get update -y
&& sudo apt-get install -y git
g++
build-essential
make
wget
RUN sudo mkdir /usr/local/cmake
&& sudo wget -O cmake-linux.sh https://cmake.org/files/v3.12/cmake-3.12.4-Linux-x86_64.sh
&& sudo sh cmake-linux.sh – --skip-license --prefix=/usr/local/cmake
ENV PATH="/usr/local/cmake/bin:{PATH}"13 RUN sudo wget https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.bz2 \ && sudo tar -xjf boost_1_68_0.tar.bz2 \ && sudo rm -rf boost_1_68_0.tar.bz2 \ && cd boost_1_68_0 \ && sudo ./bootstrap.sh --prefix=/usr/local \ && sudo ./b2 link=shared install RUN git clone https://github.com/flexferrum/jinja2cpp \ && cd jinja2cpp \ && git pull \ && git submodule -q update --init \ && mkdir build \ && cd build \ && cmake .. -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_PREFIX_PATH:PATH=/usr/local/include -DBOOST_ROOT=/usr/local/include \ && cmake --build . --target all --config Release\ && cmake --build . --target install --config Release RUN git clone https://github.com/nlohmann/json RUN g++ -std=c++17 -Wall -pedantic -D variant_CONFIG_SELECT_VARIANT=variant_VARIANT_NONSTD -I ./jinja2cpp/install/include -I ./json/single_include -I /usr/local/include/ currency_converter.cpp -L ./jinja2cpp/install/lib/static -ljinja2cpp -L usr/local/lib/ -lboost_system -lpthread -o currency_converter EXPOSE 8080 ENV apikey="<apikey>:{apikey}"
ENV accesskey=":${accesskey}"
RUN echo -e “\n/usr/local/lib” | sudo tee -a /etc/ld.so.conf
&& sudo ldconfig
RUN chmod -r–r--r-- styles.css scripts.js index.html
&& chmod a+x currency_converter
CMD ["./currency_converter", “0.0.0.0”, “8080”, “.”]

Anyone have any idea how to fix this?


(Jica) #3

Have you tried setting user osman at the end of Dockerfile? You can execute everything with root and finally execute chown -R osman.osman /home/osman and then set USER osman just before setting the ENTRYPOINT .


(Dragonosman) #4

Yeah, thanks for the reply. I was able to fix it by using user “Osman”'s home directory as the working directory. But then I decided to keep root user and just switched to normal user at the end, right before the CMD part.


(Dragonosman) #5

Although I was able to fix that other problem, I can’t actually run the container correctly. I get the output I expect from the executable, but I can’t to the app even from Curl. I don’t have port-forwarding set up on my computer, so I can’t look at in my browser anyway, but I was able to do it using Curl before at least (I’d get the HTML of my index.html file printed to the Command Prompt window). I can run application itself just fine, both on my own computer and also in my Ubuntu VM that I have in VirtualBox, though. I can visit on localhost with the given port number.

Heroku needs to be able to bind the app to $PORT to be able to run it as a Heroku app, but I can’t do it with my app. The Heroku logs say they were unable to bind it to $PORT within 60 seconds of launch and as such had to kill it with SIGKILL. This is my current Dockerfile:

FROM ubuntu:latest
WORKDIR /app
COPY scripts /app/scripts
COPY styles /app/styles
COPY index.html /app/index.html
COPY currency_converter /app/currency_converter
COPY libboost_system.so.1.68.0 /usr/local/lib/libboost_system.so.1.68.0
ENV LD_LIBRARY_PATH="/usr/local/lib/:{LD_LIBRARY_PATH}" ENV PORT 8080 ENV apikey="<apikey>:+{apikey}"
ENV accesskey=":+{accesskey}" RUN cd /app \ && chmod -r--r--r-- ./styles/styles.css ./scripts/scripts.js index.html RUN useradd -m osman USER osman CMD ["./currency_converter", "0.0.0.0", "{PORT}", “.”]

I copied the executable I’d created in my Ubuntu VM into my local machine and put it into the Docker container via the Dockerfile. I’ve pushed the image to “dragonosman/currency_converter”, which is a public repository. Any help would be appreciated.

I’m also going to try to write the same Dockerfile I had before again where I was installing the stuff I needed into the container itself and compiling the program there (only this time I’ll remove all of the stuff I wouldn’t need anymore after the executable program has been created).

Edit: Please note that in the actual Dockerfile I have on disk, the ENV commands for setting “LD_LIBRARY_PATH” and $PORT are on separate lines. I don’t know why they’re on the same line here.


(Jica) #6

I think you have to use ${PORT}.

Also, you should EXPOSE ${PORT}.


(Dragonosman) #7

I read on the Heroku docs that they actually don’t use EXPOSE. They use $PORT in a way that it overrides the EXPOSE value, if I understand correctly.

So you mean ${PORT} in the CMD command, right?


(Jica) #8

I don’t know what the CMD does, I guess that it will start a service that will listen on port ${PORT}.

I mean that you should use
EXPOSE {PORT} On the dockerfile to tell docker that the container will be listening on {PORT}, this way, it will be easier to run the container and set the port-forwarding from the host to the container. Anyway, at this moment, expose is not mandatory and if there is a service listening, docker will allow the communication with/without EXPOSE


(Dragonosman) #9

That CMD command is given to my executable program which expects an address, a port, and a docroot as command line arguments since it’s a web server.

Also, do you mean like this?

ENV PORT 8080
EXPOSE ${PORT}


(Dragonosman) #10

I’m trying to run the container locally like this:

docker run -it -p 8080:80 -e 12345 currency_converter

And the output from the executable inside the container is:

Starting server at 0.0.0.0:0…

I tried to get it to it with curl. Here’s the result:

C:\Users\Osman>curl 0.0.0.0:80/
curl: (7) Failed to connect to 0.0.0.0 port 80: Address not available

C:\Users<name>>curl 192.168.99.100:80
curl: (7) Failed to connect to 192.168.99.100 port 80: Connection refused

C:\Users<name>>curl 192.168.99.100:8080
curl: (7) Failed to connect to 192.168.99.100 port 8080: Connection refused

C:\Users<name>>curl 192.168.99.100:12345
curl: (7) Failed to connect to 192.168.99.100 port 12345: Connection refused

C:\Users<name>>curl 192.168.99.100:0
curl: (7) Couldn’t connect to server

I’ve put “EXPOSE ${PORT}” inside the Dockerfile but I took out the ENV command that’s setting the port since I think it should take it from the command line (how should I do that, though? Is just the -e flag and this EXPOSE command enough?).

And yeah, again, the CMD command at the end is given to my executable program which expects an address, a port, and a docroot as command line arguments since it’s a web server.


(Jica) #11

The command needs PORT to know where to listen

On the other hand, you shold also set the expose so docker knows which ports are available.

I think you should put both, the parameter of the command and the expose.


(Dragonosman) #12

If I do “EXPOSE ${PORT}”, will it take the port from the command line that’s entered with the “-e” flag?

Edit: I ran with -e PORT=12345 and I’ve also put “8080” as the port in the CMD command while the EXPOSE command is setting ${PORT}. But I still have the same result as before when I try to Curl it. Am I still doing something wrong?

Code is here. I also have both compiled executables, the .exe and the Linux one, on there. I also updated the Dockerfile:

FROM ubuntu:latest
WORKDIR /app
COPY scripts /app/scripts
COPY styles /app/styles
COPY index.html /app/index.html
COPY currency_converter.cpp /app/currency_converter.cpp
RUN apt-get update -y
&& apt-get install -y git g++ build-essential make wget
RUN wget https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.bz2
&& tar -xjf boost_1_68_0.tar.bz2
&& rm -r -f boost_1_68_0.tar.bz2
&& cd boost_1_68_0
&& ./bootstrap.sh
&& ./b2 install
RUN cd /usr/local/lib
&& ls
RUN wget https://cmake.org/files/v3.12/cmake-3.12.4.tar.gz
&& tar -xzvf cmake-3.12.4.tar.gz
&& rm -r -f cmake-3.12.4.tar.gz
&& cd cmake-3.12.4
&& ./bootstrap
&& make -j4
&& make install
&& cmake --version
RUN git clone https://github.com/flexferrum/jinja2cpp.git
&& cd jinja2cpp
&& git submodule -q update --init
&& mkdir build
&& cd build
&& cmake … -DCMAKE_INSTALL_PREFIX=…/install -DCMAKE_PREFIX_PATH:PATH=/usr/local/include -DBOOST_ROOT=/usr/local/include
&& cmake --build . --target all
&& cmake --build . --target install
RUN git clone https://github.com/nlohmann/json
RUN g++ -std=c++17 -Wall -pedantic -D variant_CONFIG_SELECT_VARIANT=variant_VARIANT_NONSTD -I ./jinja2cpp/install/include -I ./json/single_include -I /usr/local/include/ currency_converter.cpp -L ./jinja2cpp/install/lib/static -ljinja2cpp -L usr/local/lib/ -lboost_system -lpthread -o currency_converter
RUN cd /usr/local/
&& rm -r -f include
&& mkdir include
&& cd /app/
&& rm -r -f jinja2cpp json cmake-3.12.4 boost_1_68_0
&& rm -f currency_converter.cpp
RUN apt-get --purge remove -y git g++ build-essential make wget
&& apt-get autoremove -y
ENV LD_LIBRARY_PATH="/usr/local/lib/:{LD_LIBRARY_PATH}" EXPOSE {PORT}
ENV apikey=“apikey:+{apikey}" ENV accesskey="accesskey:+{accesskey}”
RUN cd /app
&& chmod -r–r--r-- ./styles/styles.css ./scripts/scripts.js index.html
RUN useradd -m osman
USER osman
CMD ["./currency_converter", “0.0.0.0”, “8080”, “.”]

I have “EXPOSE ${PORT}” in the actual Dockerfile as the EXPOSE command, and the ENV and EXPOSE are each on separate lines. They got mixed up here and some of the curly braces or Dollar signs got taken out.


(Dragonosman) #13

Heroku logs:

2018-11-10T01:00:32.904761+00:00 heroku[web.1]: State changed from crashed to starting
2018-11-10T01:00:35.829783+00:00 heroku[web.1]: Starting process with command ./currency_converter 0.0.0.0 \54992 .
2018-11-10T01:01:35.981969+00:00 heroku[web.1]: State changed from starting to crashed
2018-11-10T01:01:35.890316+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2018-11-10T01:01:35.890437+00:00 heroku[web.1]: Stopping process with SIGKILL
2018-11-10T01:01:35.967253+00:00 heroku[web.1]: Process exited with status 137
2018-11-10T07:00:52.742780+00:00 heroku[web.1]: State changed from crashed to starting
2018-11-10T07:00:55.199391+00:00 heroku[web.1]: Starting process with command ./currency_converter 0.0.0.0 \33374 .
2018-11-10T07:01:55.777629+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2018-11-10T07:01:55.777777+00:00 heroku[web.1]: Stopping process with SIGKILL
2018-11-10T07:01:55.910398+00:00 heroku[web.1]: Process exited with status 137
2018-11-10T07:01:55.923105+00:00 heroku[web.1]: State changed from starting to crashed
2018-11-10T12:45:17.116848+00:00 heroku[web.1]: State changed from crashed to starting
2018-11-10T12:45:20.617630+00:00 heroku[web.1]: Starting process with command ./currency_converter 0.0.0.0 \55983 .
2018-11-10T12:46:20.937629+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch
2018-11-10T12:46:20.937769+00:00 heroku[web.1]: Stopping process with SIGKILL
2018-11-10T12:46:21.070112+00:00 heroku[web.1]: State changed from starting to crashed
2018-11-10T12:46:21.051550+00:00 heroku[web.1]: Process exited with status 137

When I tried to run it like this: “docker run -e PORT=12345 -p 8080:80 -it currency_converter”, the result was:

/bin/sh: 1: [./currency_converter,: not found

And also when running it like this: “docker run -it -e PORT=12345 -p 8080:80 currency_converter”, same thing.

The CMD command had a “” in front of the PORT variable. That may have been what caused it. After taking it out I can run the container, but I still get the same “Connection refused” error when I try to Curl it.

I can run it locally without containerization just fine (.exe file built using VS2017), and I can also do it in my Ubuntu VM. It’s just with containerization using Docker that it’s not working.


(Siaarzh) #14

Osman, don’t share this image on Docker Hub, or at least make it private.
Your API’s are showing (e.g.: your google API is “…TH5FgDo”)

Since you’re using ENV VARS to store secrets, you might as well set them in your Heroku app configs and get rid of them in the Dockerfile. The app will then pick them up anyway.

Also, it doesn’t matter what you set your EXPOSE port to, because Heroku does not support it in its’ container runtime. Instead, just make sure your app listens on HTTP port:

CMD ["./currency_converter", "0.0.0.0", "80", "."]

As for local testing, the correct command for should therefore be:

$ docker container run -d \
-p 5000:80 \
-e accesskey=<your_currency_api_key> \
-e apikey=<your_gcp_api_key> \
currency_converter

I don’t know C++ so I can’t help you with your code.