Rsync error only inside docker image

General
1

Hi,

I’m using Docker version 28.0.4, build b8034c0 on Ubuntu 22 lts. I have my image done and working and now I’m trying to rsync from the image to a server. I do this twice, from local-one to remote-one and again from local-two to remote-two. Syncing from local-one to remote-one always works, but syncing from local-two to remote-two fails with an rsync error. I only get this error within the image as part of ci-cd with gitlab. So could be docker, could be rsync, might even be gitlab(but probably not). All gitlab does is run a script in the image.

This works:
rsync -avz --delete --exclude-from="./exclude-one.txt" -e "ssh -oStrictHostKeyChecking=no -i /path/to/key.ppk" --progress /local-one/ user@server.com:/remote-one

This does not:
rsync -avz --delete --exclude-from="./exclude-two.txt" -e "ssh -oStrictHostKeyChecking=no -i /path/to/key.ppk" --progress /local-two/ user@server.com:/remote-two

The error:

rsync: [client] failed to open exclude file "./exclude-two.txt": No such file or directory (2)
rsync error: error in file IO (code 11) at exclude.c(1482) [client=3.2.7]

At first glance, it looks like the file is missing, except that immediately before the rsync and immediately after the rsync I cat the file. In the output of the script I can see the contents of the file and it is correct.

If I do docker run -ts myimage:tag and copy and paste each command from the script to the command line. everything runs just fine. There’s no io error. It’s only within the docker image running w/in gitlab.

Has anyone run into something similar before?

2

You cannot run anything in an image as an image is a read-only template. You are working with containers. I just leave it here so we can use the right terms. Otherwise it could be confusing. It took me some seconds to realize what you werw writing about.

If rsync says the file is missing, I’m pretty sure it does missing from rsync’s point of view. Maybe there is a character in the filename that looks the same but it is different in binary. Or you run the commands in a different folder. You could try renaming the file to something shorter to give less chance for typos and syy if rsync finds that.

If you still need help with this, can you share how you exactly create the container, test the filename before and after running rsync?

By the way, if you really wan to copy files that are in an image, you could also create the container without starting it, and use docker cp to copy the files to the host (to a Linux filesystem). Than you could run rsync on the host and see if it has anything to do with Docker or just about the filenames.

I guess you could also use docker image export to export the filesystem which is similar to copying files from a container, except you have the whole image filesystem as a tar file which you can extract. Of course it depends on the image size. I would probably not copy 100 gigabytes to my host to rync 10 kilobytes.