nagper
November 10, 2023, 1:14pm
1
Hi,
There is a situation my knowledge is not enough to explain for.
There are 2 servers with docker installed.
On the 1st server i issue:
u753576@LSYH-u753576l1:~$ id
uid=1000(u753576) gid=1000(u753576) groups=1000(u753576),4(adm),20(dialout),24(cdrom),27(sudo),30(dip),46(plugdev),108(kvm),120(lpadmin),132(lxd),133(sambashare),145(libvirt),146(libvirt-dnsmasq),998(docker),1002(uinput),64055(libvirt-qemu)
u753576@LSYH-u753576l1:~$ docker run -it --rm redhat/ubi8-minimal:8.4-212 ls -l /
total 52
lrwxrwxrwx 1 root root 7 Apr 23 2020 bin -> usr/bin
dr-xr-xr-x 2 root root 4096 Apr 23 2020 boot
drwxr-xr-x 5 root root 360 Nov 10 13:05 dev
drwxr-xr-x 1 root root 4096 Nov 10 13:05 etc
drwxr-xr-x 2 root root 4096 Apr 23 2020 home
lrwxrwxrwx 1 root root 7 Apr 23 2020 lib -> usr/lib
lrwxrwxrwx 1 root root 9 Apr 23 2020 lib64 -> usr/lib64
drwx------ 2 root root 4096 Oct 26 2021 lost+found
drwxr-xr-x 2 root root 4096 Apr 23 2020 media
drwxr-xr-x 2 root root 4096 Apr 23 2020 mnt
drwxr-xr-x 2 root root 4096 Apr 23 2020 opt
dr-xr-xr-x 522 root root 0 Nov 10 13:05 proc
dr-xr-x--- 1 root root 4096 Oct 26 2021 root
drwxr-xr-x 1 root root 4096 Oct 26 2021 run
lrwxrwxrwx 1 root root 8 Apr 23 2020 sbin -> usr/sbin
drwxr-xr-x 2 root root 4096 Apr 23 2020 srv
dr-xr-xr-x 13 root root 0 Nov 10 13:05 sys
drwxrwxrwt 1 root root 4096 Oct 26 2021 tmp
drwxr-xr-x 12 root root 4096 Oct 26 2021 usr
drwxr-xr-x 1 root root 4096 Oct 26 2021 var
note that the majority of the files/directories are owned by root.
on the second server:
nl-sec-q01:/home/iocc~$ id
uid=1004(iocc) gid=1003(nl) groups=1003(nl),985(docker)
nl-sec-q01:/home/iocc~$ docker run -it --rm redhat/ubi8-minimal:8.4-212 ls -l /
total 0
lrwxrwxrwx 1 1004 1003 7 Apr 23 2020 bin -> usr/bin
dr-xr-xr-x 2 1004 1003 6 Apr 23 2020 boot
drwxr-xr-x 5 root root 360 Nov 10 12:50 dev
drwxr-xr-x 1 1004 1003 66 Nov 10 12:50 etc
drwxr-xr-x 2 1004 1003 6 Apr 23 2020 home
lrwxrwxrwx 1 1004 1003 7 Apr 23 2020 lib -> usr/lib
lrwxrwxrwx 1 1004 1003 9 Apr 23 2020 lib64 -> usr/lib64
drwx------ 2 1004 1003 6 Oct 26 2021 lost+found
drwxr-xr-x 2 1004 1003 6 Apr 23 2020 media
drwxr-xr-x 2 1004 1003 6 Apr 23 2020 mnt
drwxr-xr-x 2 1004 1003 6 Apr 23 2020 opt
dr-xr-xr-x 195 root root 0 Nov 10 12:50 proc
dr-xr-x--- 1 1004 1003 23 Oct 26 2021 root
drwxr-xr-x 1 1004 1003 21 Oct 26 2021 run
lrwxrwxrwx 1 1004 1003 8 Apr 23 2020 sbin -> usr/sbin
drwxr-xr-x 2 1004 1003 6 Apr 23 2020 srv
dr-xr-xr-x 13 root root 0 Nov 10 12:50 sys
drwxrwxrwt 1 1004 1003 6 Oct 26 2021 tmp
drwxr-xr-x 12 1004 1003 144 Oct 26 2021 usr
drwxr-xr-x 1 1004 1003 17 Oct 26 2021 var
the files are owned by the current user.
What makes the files on the second server owned by the current user?
meyay
November 10, 2023, 7:17pm
2
Please share the output of docker version and docker info of both servers.
nagper
November 10, 2023, 8:59pm
3
server1:
nl-sec-q01:/home/iocc~$ docker version
Client: Docker Engine - Community
Version: 24.0.6
API version: 1.43
Go version: go1.20.7
Git commit: ed223bc
Built: Mon Sep 4 12:35:25 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.6
API version: 1.43 (minimum version 1.12)
Go version: go1.20.7
Git commit: 1a79695
Built: Mon Sep 4 12:34:28 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.24
GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc:
Version: 1.1.9
GitCommit: v1.1.9-0-gccaecfc
docker-init:
Version: 0.19.0
GitCommit: de40ad0
nl-sec-q01:/home/iocc~$ docker info
Client: Docker Engine - Community
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.21.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
scan: Docker Scan (Docker Inc.)
Version: v0.23.0
Path: /usr/libexec/docker/cli-plugins/docker-scan
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 1
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc version: v1.1.9-0-gccaecfc
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 3.10.0-693.21.1.el7.x86_64
Operating System: JBoss
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.64GiB
Name: nl-sec-q01
ID: 8019cf1a-dd02-4f96-8e73-db8fb3f94793
Docker Root Dir: /opt/lsy/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
server2:
u753576@LSYH-u753576l1:~$ docker version
Client: Docker Engine - Community
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:08:01 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.24
GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc:
Version: 1.1.9
GitCommit: v1.1.9-0-gccaecfc
docker-init:
Version: 0.19.0
GitCommit: de40ad0
u753576@LSYH-u753576l1:~$ docker info
Client: Docker Engine - Community
Version: 24.0.7
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.21.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 1
Running: 1
Paused: 0
Stopped: 0
Images: 45
Server Version: 24.0.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
runc version: v1.1.9-0-gccaecfc
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 5.15.0-88-generic
Operating System: Ubuntu 22.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 30.95GiB
Name: LSYH-u753576l1
ID: MS2C:FY6G:KOC3:3XA5:J2RT:5FE3:OFBX:LGHI:PZNZ:FGJT:UAQ7:FFZO
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http://proxy:3128/
HTTPS Proxy: http://proxy:3128/
No Proxy: localhost,127.0.0.1,192.168.1.1,::1,172.*.*.*,*.local
Username: nagper
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
rimelek
November 10, 2023, 10:14pm
4
I edited your post. Please help us by sharing different kind of outputs in separate code blocks. I almost thought you didn’t share the output of docker info because I couldn’t find the command.
One thing I see is that the you shared the info of the previously called “1st server” as “server2” and “2st server” as “server1” which also confused me. So let’s call them Ubuntu (LSYH-u753576l1) and JBoss (nl-sec-q01) which is shown in the docker info output. Now I have to say I have no idea what “JBoss” as an operating system is, but based on the kernel If I’m not mistaken that indicates some kind of RedHat OS.
On the JBoss OS you have an older kernel and CGroup 1 instead of CGroup 2 which I can notice as difference. I also don’t know about any supported OS called JBoss. The only JBoss I knew was a web server for Java apps.
Even if you were using usernamespaces or rootless Docker, you should see root inside the container so it looks like as if you were using usernamespaces on the JBoss OS but it couldn’t handle it and you see the same ID inside the container where it should be root. The only folders that are owned by root on JBoss are special filesystems for communication with the kernel.
On JBoss even the Dockerdata root is on different. Is it mounted from somewhere? Is that folder special in some way?
In summary, I wouldn’t be surprised if the problem was using the recent Docker on an unsupported operating system and an old kernel in combination with rootless Docker. Although I don’t see any sign of rootless Docker.
How did you installed it on JBoss?
nagper
November 11, 2023, 9:38am
5
Sorry you’re right, i did swap server1<->server2 indeed.
The “Jboss” is a “Red Hat Enterprise Linux Server release 7.4 (Maipo)”
None of the docker installations are rootless indeed and neither of them use user namespaces.
/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
You might be right in the on RedHat 7.x this version of docker is not supported, so that could cause this behaviour.
rimelek
November 11, 2023, 10:19am
6
On RHEL only s360x architecture is supported officially:
and yours is x86_64.
Since you say you used the same command line parameters, assuming there is no other configuration (/etc/docker/daemon.json)` that release could be installed from binaries:
Whicih means the installer (or you manually) had to configure the host, the kernel, everything to make it work and I have never done it.
nagper
November 11, 2023, 10:26am
7
All right, thanks for your support.