Fail to create docker-container builder

General
1

Hi,

I was successful in creating a Docker container driver builder on Ubuntu 20.04, which runs under WSL 2.0, using a custom configuration file. Here are my commands:

docker buildx create --name docker-container-driver-builder \
--driver docker-container \
--driver-opt image=<some image name> \
--driver-opt default-load=true \
--config //<full_path_to>.toml

docker buildx inspect --bootstrap --builder docker-container-driver-builder

The .toml file contains only two lines:
[worker.oci]
max-parallelism = 3

When trying to run on a remote Ubuntu 20.04 VM, I encounter a failure on the second inspect command. It seems there is an issue with the file being copied into the driver’s container:


> docker-container-driver-builder
> [+] Building 16.3s (1/1) FINISHED                                                                                                                                                                                                                       
>  => ERROR [internal] booting buildkit                                                                                                                                                                                                            16.3s
>  => => pulling image amr-registry.caas.intel.com/tpe-dev/devsecops/builder:initial                                                                                                                                                                1.0s
>  => => creating container buildx_buildkit_docker-container-driver-athena-builder0                                                                                                                                                                15.2s
> ------                                                                                                                                                                                                                                                  
>  > [internal] booting buildkit:                                                                                                                                                                                                                        
> 16.26 failed to load config from /etc/buildkit / buildkitd.toml
> 16.26 7failed to load config from /etc/buildkit / buildkitd.toml
> 16.26   /src/cmd/buildkitd/config/load.go:32
> 16.26 main.main /src/cmd/buildkitd/config/load.go:32
> 16.26   /src/cmd/buildk /src/cmd/buildkitd/main.go:230
> 16.26 github.com/urfave/cli.(*App).Run
> 16.26 . /srcgithub.com/urfave/cli.(*App).Run
> 16.26 main.main
> 16.26   /src/cmd/buildkitd/main.go:main.main
> 16.26 runtime.main
> 16.26 ' /usr/localruntime.main
> 16.26 runtime.goexit
> 16.26 , /usr/local/go/sruntime.goexit
> 16.26 [rootlesskit:child ] error: command [buildki[rootlesskit:child ] error: command [buildkitd --config /etc/buildkit/buildkitd.toml --allow-insecure-entitlement=network.host] exited: exit status 1
> 16.26 buildkitd: open /etc/buildkit/buildkitd.toml: permissionbuildkitd: open /etc/buildkit/buildkitd.toml: permission denied
> 16.26 github.com/moby/buildkit/cmd/buildkitd/config.LoadFile
> 16.26 github.com/moby/buildkit/cmd/buildkitd/config.LoadFile
> 16.26 main.main.func3
> 16.26   /src/cmd/builmain.main.func3
> 16.26 github.com/urfave/cli.HandleActigithub.com/urfave/cli.HandleAction
> 16.26 github.com/urfave/cli.(*App).Run
> 16.26 . /srcgithub.com/urfave/cli.(*App).Run
> 16.26 main.main
> 16.26   /src/cmd/buildkitd/main.go:main.main
> 16.26 runtime.main
> 16.26 ' /usr/localruntime.main
> 16.26 runtime.goexit
> 16.26 , /usr/local/go/sruntime.goexit
> 16.26 [rootlesskit:child ] error: command [buildki[rootlesskit:child ] error: command [buildkitd --config /etc/buildkit/buildkitd.toml --allow-insecure-entitlement=network.host] exited: exit status 1
> 16.26 buildkitd: open /etc/buildkit/buildkitd.toml: permissionbuildkitd: open /etc/buildkit/buildkitd.toml: permission denied
> 16.26 github.com/moby/buildkit/cmd/buildkitd/config.LoadFile
> 16.26 github.com/moby/buildkit/cmd/buildkitd/config.LoadFile
> 16.26 main.main.func3
> 16.26   /src/cmd/builmain.main.func3
> 16.26 github.com/urfave/cli.HandleActigithub.com/urfave/cli.HandleAction
> 16.26 github.com/urfave/cli.(*App).Run
> 16.26 . /srcgithub.com/urfave/cli.(*App).Run
> 16.26 main.main
> 16.26   /src/cmd/buildkitd/main.go:main.main
> 16.26 runtime.main
> 16.26 ' /usr/localruntime.main
> 16.26 runtime.goexit
> 16.26 , /usr/local/go/sruntime.goexit
> 16.26 [rootlesskit:parent] error: child exited: exit status 1
> 16.26 @buildkitd: open /etc/buildkit/buildkitd.toml: permission denied
> 16.26 8failed to load[rootlesskit:parent] error: child exited: exit status 1
> 16.26 failed to load config from /etc/buildkit/buildkitd.toml
> 16.26 7failed to load config from /etc/buildkit/buildkitd.toml
> 16.26   /src/cmd/buildkitd/config/load.go:32
> 16.26 main.main /src/cmd/buildkitd/config/load.go:32
> 16.26   /src/cmd/buildk /src/cmd/buildkitd/main.go:230
> 16.26   /src/vendor/github.com/urfave/cli/      /src/vendor/github.com/urfave/cli/app.go:524
> 16.26   /src/vendor/github.com/urfave/cl        /src/vendor/github.com/urfave/cli/app.go:286
> 16.26   /src/cmd/       /src/cmd/buildkitd/main.go:378
> 16.26   /usr/local/g    /usr/local/go/src/runtime/proc.go:267
> 16.26   /usr/local/go/  /usr/local/go/src/runtime/asm_amd64.s:1650
> 16.26 [rootlesskit:parent] error: child exited: exit status 1
> 16.26 @buildkitd: open /etc/buildkit/buildkitd.toml: permission denied
> 16.26 8failed to load[rootlesskit:parent] error: child exited: exit status 1
> 16.26 failed to load config from /etc/buildkit/buildkitd.toml
> 16.26 7failed to load config from /etc/buildkit/buildkitd.toml
> 16.26   /src/cmd/buildkitd/config/load.go:32
> 16.26 main.main /src/cmd/buildkitd/config/load.go:32
> 16.26   /src/cmd/buildk /src/cmd/buildkitd/main.go:230
> 16.26   /src/vendor/github.com/urfave/cli/      /src/vendor/github.com/urfave/cli/app.go:524
> 16.26   /src/vendor/github.com/urfave/cl        /sr

I have checked permissions/ownership issues, and the fil path provided looks fine. I’ve also verified all configuration files. Tried running on both machines with Docker version 25.04 (client and server) and the latest version, 27.3.1. The issue occurs with both buildx plugin version 0.16.2 and after upgrading to 18.0 (latest). no issues on the WSL machine, and there is a consistent issue on the ubuntu vm 20.04. This can happen on multiple machines with the same attributes.

Any ideas on how to resolve this issue?

5

Please share the output of these commands:

docker info
ls -lZ /etc/buildkit
8

sorry for the late reply. Here is the output:

=> docker info
Client:
Version: 27.3.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.19.3
Path: /mnt/sdb/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.2
Path: /mnt/sdb/.docker/cli-plugins/docker-compose

Server:
Containers: 6
Running: 5
Paused: 0
Stopped: 1
Images: 14
Server Version: 27.3.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
runc version: v1.1.14-0-g2c9f560
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 5.15.0-130-generic
Operating System: Ubuntu 22.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 31.39GiB
Name: Dev-vm-14
ID: 6d91c263-1a0a-4845-b303-0678b563dff4
Docker Root Dir: /mnt/sdb/docker-data
Debug Mode: true
File Descriptors: 69
Goroutines: 89
System Time: 2025-01-26T06:13:43.18758543-08:00
EventsListeners: 0
HTTPS Proxy: http://proxy-dmz.intel.com:912
No Proxy: intel.com,localhost,127.0.0.1,::1
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://amr-registry.caas.intel.com/
Live Restore Enabled: false
Product License: Community Engine

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

And for the directory content - it is empty. Even when non empty, with some buildkitd.toml file it still happens.