Hi all,
I’m new to the forum and docker swarm; relatively new to docker in general. I’ve been struggling with an seemingly simple set-up for several days now. There is probably a rookie mistake hidden in here, but I can’t get my head around it and I’m looking forward to your help!
Issue: I can’t get docker stack deploy to work with the environment runtime argument. I can get the configuration to work with docker-compose.
The configuration that I’m sharing is based on MySQL docker documentation. From what I can tell so far, the issue is not related to MySQL.
Contents of docker-compose.yml:
version: '3.8'
services:
db:
image: mysql:latest
command: --default-authentication-plugin=mysql_native_password
environment:
- MYSQL_ROOT_PASSWORD=123
adminer:
image: adminer
ports:
- 8080:8080
networks:
default:
name: db_network
driver: overlay
Actual behavior:
$ sudo docker stack deploy -c docker-compose.yml mysql-tester
Creating network db_network
Creating service mysql-tester_adminer
Creating service mysql-tester_db
$ sudo docker stack ps mysql-tester
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ux3wysxllr3e mysql-tester_adminer.1 adminer:latest NAS Running Running 55 seconds ago
nd165ye84yfs mysql-tester_db.1 mysql:latest NAS Running Starting 1 second ago
pzko4nzra8fk \_ mysql-tester_db.1 mysql:latest NAS Shutdown Failed 6 seconds ago "task: non-zero exit (1)"
yyxwor45t6jk \_ mysql-tester_db.1 mysql:latest NAS Shutdown Failed 18 seconds ago "task: non-zero exit (1)"
izddw0mvn0e9 \_ mysql-tester_db.1 mysql:latest NAS Shutdown Failed 30 seconds ago "task: non-zero exit (1)"
u21trxihjmas \_ mysql-tester_db.1 mysql:latest NAS Shutdown Failed 42 seconds ago "task: non-zero exit (1)"
$ # the following two command only work if a new container is started and not yet crashed -- in practice I only get output once in every few tries of the same command
$ sudo docker logs $(sudo docker ps -q --filter name=mysql-tester_db)
2021-08-25 19:35:26+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
2021-08-25 19:35:28+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-08-25 19:35:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
2021-08-25 19:35:28+00:00 [ERROR] [Entrypoint]: Database is uninitialized and password option is not specified
You need to specify one of the following:
- MYSQL_ROOT_PASSWORD
- MYSQL_ALLOW_EMPTY_PASSWORD
- MYSQL_RANDOM_ROOT_PASSWORD
From docker inspect it appears that the Config.Cmd reflects my docker-compose.yml settings, but the Config.Env does not.
docker inspect
$ sudo docker inspect $(sudo docker ps -q --filter name=mysql-tester_db)
[
{
"Id": "85d6b382eb13529439484a259cbb69dddcb5835943564d263549bfbd124ae164",
"Created": "2021-08-25T19:36:30.67405673Z",
"Path": "docker-entrypoint.sh",
"Args": [
"--default-authentication-plugin=mysql_native_password"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 9224,
"ExitCode": 0,
"Error": "",
"StartedAt": "2021-08-25T19:36:34.912297707Z",
"FinishedAt": "0001-01-01T00:00:00Z",
"StartedTs": 1629920194,
"FinishedTs": -62135596800
},
"Image": "sha256:5a4e492065c722ec8cc7413552bafc6fd5434c5ad90797e898ccc4e347e21aa5",
"ResolvConfPath": "/volume1/@docker/containers/85d6b382eb13529439484a259cbb69dddcb5835943564d263549bfbd124ae164/resolv.conf",
"HostnamePath": "/volume1/@docker/containers/85d6b382eb13529439484a259cbb69dddcb5835943564d263549bfbd124ae164/hostname",
"HostsPath": "/volume1/@docker/containers/85d6b382eb13529439484a259cbb69dddcb5835943564d263549bfbd124ae164/hosts",
"LogPath": "/volume1/@docker/containers/85d6b382eb13529439484a259cbb69dddcb5835943564d263549bfbd124ae164/log.db",
"Name": "/mysql-tester_db.1.5lw73wrxzwppkhy26ym0vk9vh",
"RestartCount": 0,
"Driver": "btrfs",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "docker-default",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "db",
"Config": {}
},
"NetworkMode": "default",
"PortBindings": {},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GOSU_VERSION=1.12",
"MYSQL_MAJOR=8.0",
"MYSQL_VERSION=8.0.26-1debian10"
],
"ConsoleSize": [
0,
0
],
"Isolation": "default",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": [],
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": null,
"Name": "btrfs"
},
"Mounts": [
{
"Type": "volume",
"Name": "05a451cca22a94dfcffc5b73690f4803ae470e60bf97e7ff45bc9ef19e08b154",
"Source": "/volume1/@docker/volumes/05a451cca22a94dfcffc5b73690f4803ae470e60bf97e7ff45bc9ef19e08b154/_data",
"Destination": "/var/lib/mysql",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "85d6b382eb13",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"3306/tcp": {},
"33060/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GOSU_VERSION=1.12",
"MYSQL_MAJOR=8.0",
"MYSQL_VERSION=8.0.26-1debian10"
],
"Cmd": [
"--default-authentication-plugin=mysql_native_password"
],
"Image": "mysql:latest",
"Volumes": {
"/var/lib/mysql": {}
},
"WorkingDir": "",
"Entrypoint": [
"docker-entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"com.docker.stack.namespace": "mysql-tester",
"com.docker.swarm.node.id": "roe85itnvzbdesc4y1kc7387d",
"com.docker.swarm.service.id": "sc5yg9m0b0bwgrxm7qwbiy47p",
"com.docker.swarm.service.name": "mysql-tester_db",
"com.docker.swarm.task": "",
"com.docker.swarm.task.id": "5lw73wrxzwppkhy26ym0vk9vh",
"com.docker.swarm.task.name": "mysql-tester_db.1.5lw73wrxzwppkhy26ym0vk9vh"
},
"DDSM": false
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "18219b8b9ed4d07b19585c80f19c6950a9cacaddbad6fb8ba3f542d4a9c22572",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"3306/tcp": null,
"33060/tcp": null
},
"SandboxKey": "/var/run/docker/netns/18219b8b9ed4",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"db_network": {
"IPAMConfig": {
"IPv4Address": "10.0.39.38"
},
"Links": null,
"Aliases": [
"85d6b382eb13"
],
"NetworkID": "0109v22egwpfwi6dn3sguu1wc",
"EndpointID": "bfcfc9ece87d65a31191e926701a2da1fc747c0faeab869cebee69a60b957e92",
"Gateway": "",
"IPAddress": "10.0.39.38",
"IPPrefixLen": 24,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:0a:00:27:26",
"DriverOpts": null
}
}
}
}
]
Expected behavior would be that the environmental variable MYSQL_ROOT_PASSWORD is set as defined in the container.
When trying the same thing with docker-compose, everything works fine.
Expected behavior (docker compose)
$ docker-compose up -d
WARNING: The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use `docker stack deploy`.
Creating network "db_network" with driver "overlay"
Creating docker_adminer_1 ... done
Creating docker_db_1 ... done
$ # wait for several minutes
$ docker logs docker_db_1
2021-08-25 19:25:29+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
2021-08-25 19:25:31+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-08-25 19:25:31+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.26-1debian10 started.
2021-08-25 19:25:31+00:00 [Note] [Entrypoint]: Initializing database files
2021-08-25T19:25:31.984826Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.26) initializing of server in progress as process 43
2021-08-25T19:25:32.049285Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-08-25T19:25:45.424038Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-08-25T19:26:15.809887Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
2021-08-25T19:26:15.811557Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
2021-08-25T19:26:15.997156Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2021-08-25 19:27:27+00:00 [Note] [Entrypoint]: Database files initialized
2021-08-25 19:27:27+00:00 [Note] [Entrypoint]: Starting temporary server
2021-08-25T19:27:28.258888Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.26) starting as process 90
2021-08-25T19:27:28.647446Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-08-25T19:27:31.484457Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-08-25T19:27:35.195883Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
2021-08-25T19:27:35.196124Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
2021-08-25T19:27:35.261837Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-08-25T19:27:35.262681Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2021-08-25T19:27:35.313082Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-08-25T19:27:35.390931Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2021-08-25T19:27:35.391814Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.26' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
2021-08-25 19:27:35+00:00 [Note] [Entrypoint]: Temporary server started.
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2021-08-25 19:27:48+00:00 [Note] [Entrypoint]: Stopping temporary server
2021-08-25T19:27:48.723999Z 10 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.26).
2021-08-25T19:28:13.893964Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.26) MySQL Community Server - GPL.
2021-08-25 19:28:14+00:00 [Note] [Entrypoint]: Temporary server stopped
2021-08-25 19:28:14+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
2021-08-25T19:28:15.316674Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.26) starting as process 1
2021-08-25T19:28:15.414013Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2021-08-25T19:28:17.930795Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2021-08-25T19:28:20.316099Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1 is enabled for channel mysql_main
2021-08-25T19:28:20.316591Z 0 [Warning] [MY-013746] [Server] A deprecated TLS version TLSv1.1 is enabled for channel mysql_main
2021-08-25T19:28:20.496108Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2021-08-25T19:28:20.496659Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2021-08-25T19:28:20.616333Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2021-08-25T19:28:20.811543Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2021-08-25T19:28:20.818458Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.26' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
I’ve tried the same configuration with docker secrets, but that leads to the same observations.
Configuration with docker secrets
Setting secret:
$ echo MYSQL_ROOT_PASSWORD=123 >> ./mysql_root_password.txt
$ cat mysql_root_password.txt
MYSQL_ROOT_PASSWORD=123
Updating docker-compose.yml:
version: '3.8'
secrets:
MYSQL_ROOT_PASSWORD:
file: ./mysql_root_password.txt
services:
db:
image: mysql:latest
command: --default-authentication-plugin=mysql_native_password
environment:
- MYSQL_ROOT_PASSWORD_FILE=/run/secrets/MYSQL_ROOT_PASSWORD
secrets:
- MYSQL_ROOT_PASSWORD
adminer:
image: adminer
ports:
- 8080:8080
networks:
default:
name: db_network
driver: overlay
With this setup, the results are exactly the same.
Versions/builds (Synology NAS, linux based):
$ sudo docker-compose version
docker-compose version 1.28.5, build 324b023a
docker-py version: 4.4.4
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
$ sudo docker version
Client:
Version: 20.10.3
API version: 1.41
Go version: go1.15.6
Git commit: b455053
Built: Mon Jun 21 02:03:40 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.3
API version: 1.41 (minimum version 1.12)
Go version: go1.15.6
Git commit: a3bc36f
Built: Mon Jun 21 02:05:07 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.4.3
GitCommit: ea3508454ff2268c32720eb4d2fc9816d6f75f88
runc:
Version: v1.0.0-rc93
GitCommit: 31cc25f16f5eba4d0f53e35374532873744f4b31
docker-init:
Version: 0.19.0
GitCommit: ed96d00
I have searched the web quite extensively, but perhaps I’m searching with the wrong keywords (usually end up in issues that involve passing environment variables from host to docker, e.g. VAR=$(VAR)). I’ll post some of my keywords here so that others may find it: docker stack swarm environment variable ignored etc.
Help is very much appreciated!