Docker Community Forums

Share and learn in the Docker community.

Issue with rsync via Dockerfile

docker

(Blitzor) #1

Hi guys!
I think I’m just stupid but I dont get it why it dosent work.

My Problem:
I have my dockerfile with aclone.sh
Theclone.sh will just be copied in the workdir and it works.

But at the build moment if it comes to the 2nd last line:
RUN ./clone.sh
It copies everything via rsync from a imageserver to the container.
The output is correct and the data transfer is sucessfull.

rsync -avzbe "ssh -i mykey -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 22"  imageserver@127.0.0.1:/path/to/my/files/* . 2>&1

After this appears this:

sent 885,567 bytes  received 2,730,495,466 bytes  10,903,716.70 bytes/sec
total size is 6,037,328,207  speedup is 2.21
 ---> 20d5a26839f6

So it was sucessfull.

In the last line of the dockerfile is a RUN ls and it shows only the clone.sh.
If I just start a container with this image, only theclone.sh there.

And now it comes crazy:
If I execute the clone.sh in the container self via docker exec -it mycontainer bash and then ./clone.sh it works without any problem and the files are there.

So TL;DR Version:

Dockerfile rsync dosent work. No files are there
Via manual container exec did it work.


(David Maze) #2

A pointer to your Dockerfile and the clone.sh script might help give some pointers.

Reading the post, though, I’d have three big concerns:

  1. It looks like you’re baking an ssh key into your image. That key is there to be stolen and reused by anyone who gets the image later.
  2. If you change the contents of the files elsewhere, then re-run docker build, the Docker layer caching knows that it already ran the clone.sh script but not anything about its dependencies, and so it won’t repeat it.
  3. A 6 GB Docker image is unwieldy, to say the least (and this process produces a single 6 GB layer which makes things even harder to work with).

If you really need this in a single image, a better path would be to copy the data locally before running docker build and then just let the normal COPY or ADD commands copy it in. This will be slow, but it will be reliable and it doesn’t risk leaking secrets.


(Blitzor) #3

First:
Thanks for your answer!

All right! I will change this. Sorry I’m a beginner :smiley:

Edit:
One more thing:
It didn’t even work if i put in the dockerfile:
RUN rsync mystuff .
But in the Container it works without any problem :frowning:

"If you change the contents of the files elsewhere, then re-run docker build, the Docker layer caching knows that it already ran the clone.sh script but not anything about its dependencies, and so it won’t repeat it."
So it would work if i rename it or? Is there a way to delete the cache?

The reason for this big image is this:
We have on our imageserver gameserver images smth like ark and so one.
We would like to create a docker container which already has from beginning the ark gameserver files in it.
So the problem is:
If we copy it manually, we need to : Download the Image to our docker host, & copy it into the machine at building. Thats why we want to download it instantly.