Docker for Mac Version: 17.09.1-ce-mac42 (21090)
OS X: 10.13.3
I have created a container using docker-compose (and also via dockerode). In both cases, the container gets created and started fine for sometime.
My Docker-Compose File
version: '3'
services:
ns-xxxx:
image: internalhost/db:master
container_name: xxxx
expose:
- "27017"
ports:
- "27017:27017"
restart: always
volumes:
- ./shared:/data/shared
entrypoint: /data/shared/entrypoint.sh
I able to access the container-services (example: MongoDB based on the above compose-file) from the Host (MacOS). Say after sometime say 15mins or so, I am unable to access the container services from the Host(MacOS). Ie if i try to connect to mongodb via a mongo-client - I get error Network is Unreachable.
The interesting thing is this: the container is still active and healthy and responding if I do docker ps - a
as well as I do
docker exec -it my-container-name /bin/bash
$mongo // inside the container - mongo-services works fine too as its able to connect to MongoDB
Also if I try to check if the host/port responding like the below command in the HOST (macOS):
nc -vz -w 1 localhost 27017
found 0 associations
found 1 connections:
1: flags=82<CONNECTED,PREFERRED>
outif lo0
src ::1 port 57605
dst ::1 port 27017
rank info not available
TCP aux info available
Connection to localhost port 27017 [tcp/*] succeeded!
Unsure what the issue is or where to look… as it works for the first few mins
PS: I have an office MAC with corp firewall (not sure if it has anything to do with the networking of the docker daemon)
My docker daemon logs:
&types.Event{Type:\"exit\", Id:\"cea655fd22c6531c22799e38e47be61a28b9967923893a823afc0bfa87a56213\", Status:0x0, Pid:\"3f767e2e969ce1a04a5148f3ece400f52cd3223c23f08c49948b11da69d1ca62\", Timestamp:(*timestamp.Timestamp)(0xc4215d96c0)}"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610503135Z" level=debug msg="attach: stderr: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610526005Z" level=debug msg="attach: stdout: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610580876Z" level=debug msg="Health check for container 84373b21f3ad4a915dc87303a0a3e356a770da01642495325014e46328778753 done (exitCode=0)"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610631303Z" level=debug msg="libcontainerd: received containerd event: &types.Event{Type:\"exit\", Id:\"dca5fa7093590e9558053d8d3aae9430eb7bcb90042e7b6e5fabbc8660c73033\", Status:0x0, Pid:\"dd75386e692b5814cc88ef5a9070c24b51fbc57a1de8bfc3d3cb70402e5a533e\", Timestamp:(*timestamp.Timestamp)(0xc4215d9830)}"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610704721Z" level=debug msg="attach: stderr: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610714181Z" level=debug msg="attach: stdout: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.610729097Z" level=debug msg="Health check for container cea655fd22c6531c22799e38e47be61a28b9967923893a823afc0bfa87a56213 done (exitCode=0)"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.611490050Z" level=debug msg="attach: stderr: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.611551629Z" level=debug msg="attach: stdout: end"
Feb 15 21:54:09 moby root: time="2018-02-15T21:54:09.611566992Z" level=debug msg="Health check for container dca5fa7093590e9558053d8d3aae9430eb7bcb90042e7b6e5fabbc8660c73033 done (exitCode=0)"
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.364190087Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.364367057Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:45950, forwarding to udp:192.168.65.1"
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.370744338Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.579898294Z" level=debug msg="Name To resolve: x1-some-fake-host."
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.579988909Z" level=debug msg="[resolver] lookup name x1-some-fake-host. present without IPv6 address"
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.580626038Z" level=debug msg="Name To resolve: x1-some-fake-host."
Feb 15 21:54:10 moby root: time="2018-02-15T21:54:10.580724092Z" level=debug msg="[resolver] lookup for x1-some-fake-host.: IP [172.18.0.3]"
Feb 15 21:54:11 moby root: time="2018-02-15T21:54:11.366577040Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:11 moby root: time="2018-02-15T21:54:11.366774851Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:49930, forwarding to udp:192.168.65.1"
Feb 15 21:54:11 moby root: time="2018-02-15T21:54:11.367222223Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:12 moby root: time="2018-02-15T21:54:12.367918835Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:12 moby root: time="2018-02-15T21:54:12.368099776Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:38466, forwarding to udp:192.168.65.1"
Feb 15 21:54:12 moby root: time="2018-02-15T21:54:12.368604729Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.369450214Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.369619231Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:51783, forwarding to udp:192.168.65.1"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.370095101Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586119843Z" level=debug msg="Name To resolve: x2-some-fake-host."
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586285820Z" level=debug msg="[resolver] lookup name x2-some-fake-host. present without IPv6 address"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586171325Z" level=debug msg="Name To resolve: x5-some-fake-host."
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586459758Z" level=debug msg="[resolver] lookup name x5-some-fake-host. present without IPv6 address"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586528186Z" level=debug msg="Name To resolve: x2-some-fake-host."
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.586724674Z" level=debug msg="[resolver] lookup for x2-some-fake-host.: IP [172.18.0.9]"
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.587005819Z" level=debug msg="Name To resolve: x5-some-fake-host."
Feb 15 21:54:13 moby root: time="2018-02-15T21:54:13.587072990Z" level=debug msg="[resolver] lookup for x5-some-fake-host.: IP [172.18.0.6]"
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.370378559Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.370592525Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:32948, forwarding to udp:192.168.65.1"
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.371058859Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.589264739Z" level=debug msg="Name To resolve: x3-some-fake-host."
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.589400743Z" level=debug msg="[resolver] lookup name x3-some-fake-host. present without IPv6 address"
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.589860972Z" level=debug msg="Name To resolve: x3-some-fake-host."
Feb 15 21:54:14 moby root: time="2018-02-15T21:54:14.589950607Z" level=debug msg="[resolver] lookup for x3-some-fake-host.: IP [172.18.0.11]"
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.372011095Z" level=debug msg="Name To resolve: docker.for.mac.localhost."
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.372596986Z" level=debug msg="[resolver] query docker.for.mac.localhost. (AAAA) from 172.18.0.12:53653, forwarding to udp:192.168.65.1"
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.373880782Z" level=debug msg="[resolver] received A record \"192.168.65.1\" for \"docker.for.mac.localhost.\" from udp:192.168.65.1"
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.590203435Z" level=debug msg="Name To resolve: x4-some-fake-host."
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.590376543Z" level=debug msg="[resolver] lookup name x4-some-fake-host. present without IPv6 address"
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.590320742Z" level=debug msg="Name To resolve: x6-some-fake-host."
Feb 15 21:54:15 moby root: time="2018-02-15T21:54:15.590487756Z" level=debug msg="[
docker inspect container-name
[
{
"Id": "133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0",
"Created": "2018-02-15T20:51:39.046278673Z",
"Path": "/data/shared/entrypoint.sh",
"Args": [],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 2483,
"ExitCode": 0,
"Error": "",
"StartedAt": "2018-02-15T20:51:39.841657732Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:9f9ba1d05947e0f95435e03520ef6124e1bdba44cae8fac9811ae66ee7fbf5d5",
"ResolvConfPath": "/var/docker/containers/133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0/resolv.conf",
"HostnamePath": "/var/docker/containers/133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0/hostname",
"HostsPath": "/var/docker/containers/133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0/hosts",
"LogPath": "/var/docker/containers/133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0/133f77e6d00f5afc1031928d1e0468204b1829ab10975285e4f69eb1241d74d0-json.log",
"Name": "/build-slaveqadb",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/xxxx/yyyyy/mongodb/shared:/data/shared:rw"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "containername_default",
"PortBindings": {
"27017/tcp": [
{
"HostIp": "",
"HostPort": "27017"
}
]
]
},
"RestartPolicy": {
"Name": "always",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": [],
"CapAdd": null,
"CapDrop": null,
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "shareable",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": 0,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/docker/overlay2/9911ec636156cdd8d1c9e3256587c914ea875b9ba3643faee91f6cea35824698-init/diff:/var/docker/overlay2/21753ab186c2f5087f5e1b390cf23ea23aaa209b62a5373922f44b6e74e4c8bc/diff:/var/docker/overlay2/32643bdeb1ffb425ef3c93b17752d0615b02060ef239a9191891e52c6ff45063/diff:/var/docker/overlay2/072ede27bcb7c3d8882e5dccb85f9794b3c30b693946037a35fb97fc64a2f356/diff:/var/docker/overlay2/8dbfa3c7c93e23fffa8a26a4bb89ba154b496b9ebc85cb488bd7746652338b32/diff:/var/docker/overlay2/1a0d89c74b0ec24a00333b6ddf4faf6a29af63d02d1de1c1a29de0abc88ce7c2/diff:/var/docker/overlay2/43c7c2f763e08c2735996d529a3b125fae54c9a95c7b10badabd953b9d70d354/diff:/var/docker/overlay2/00fc9994414b848a3ab3b2b0062b20a7b7f13c97fe35e4531ed7ca740f1ea24e/diff:/var/docker/overlay2/10cd148ce7245bbf0a2f1dfeb298f8e4fdef40490511cbeb48e3e40a038515c3/diff:/var/docker/overlay2/4f717b7605262c7b462a136d08712b264b855da697bb54047738a229d23f9091/diff:/var/docker/overlay2/12d845ec364c1faab03f8c653924c1f0fa9d3870370ba4d748651ad73d189758/diff:/var/docker/overlay2/4a613bd647be114d175e319d4d989083c63d2a3c0dd8a4ac8ff137ef66e8dc7d/diff:/var/docker/overlay2/5c6f650aaeda667bde37505ad586bd0c61316033050edde15eeb9a43339172df/diff:/var/docker/overlay2/a4fa7024bc7713dda5ef7feac520043b1d66a70287030eec104bb3f2317959d8/diff:/var/docker/overlay2/659e88c4869c3e37abbad3678509a75b1b63ba510775867ec07942cd180d8f7b/diff:/var/docker/overlay2/b93e4f4175021869322c9d7aa6ad86103a5e044d98a895f91a59d34b107aa812/diff:/var/docker/overlay2/6cd58d86fb7980f0b5d29f859b2efcec9d8b1f747640cbb473fe5eaf47bab110/diff:/var/docker/overlay2/c66f2cf20025340f253d57e53b12b5df87ce2ba766fe928b5b76d3655025041a/diff:/var/docker/overlay2/909053671b5e9f4d95a73b10e08b40b6fcc1581be0998c30207312b2ffbf15af/diff:/var/docker/overlay2/fff6d1761131c032e6bad383753d1d3021eed7fe60b7c9eadb762b3e45c33b58/diff:/var/docker/overlay2/db6c99767c383eb5e64bd77651c95df53b9eb6c40c1c5ac055ee1c07fdcf1486/diff:/var/docker/overlay2/22dcb5f0a3b5ff1527e6bc427fd0ad4d3f8f65642562ea9a3a1ac995fd128498/diff:/var/docker/overlay2/c5273b7e4f5f361eeb7932a1376798a9db484e9cd3e04ec44871252350f5faa3/diff:/var/docker/overlay2/99670a813929b63d21de5b4d07f9796cbcacd103e9e110f1a9824f0cd60c4003/diff:/var/docker/overlay2/12e457bd1264395ac4158bb1e89e62daafec55ee4a4554da4c83c263188f8009/diff:/var/docker/overlay2/0ef440acc9a9c55770633fc79369dacbb3fc20c91e1051d0d7d2ed9583726ab7/diff:/var/docker/overlay2/a90a94e4489d89e2e6cd616b8c44681d236ea202ab1dd01e29666f66d3ca7ef5/diff:/var/docker/overlay2/28108308dfc503368ae5f6759545fd3f1551310638baae3b2b74c8b843f69ba5/diff:/var/docker/overlay2/615a27792fa2c9d84c3e4e5e8a8d1355b9a979ee12e485350b6f2b2af5dc3918/diff:/var/docker/overlay2/e2c19ea5f841fd1e965e41a3728d9f4e9a40a44931d03f48e901616548afb47e/diff:/var/docker/overlay2/668347bea5b89f07ba47a62234b7a3b3f56f17329c4448613727961042744256/diff",
"MergedDir": "/var/docker/overlay2/9911ec636156cdd8d1c9e3256587c914ea875b9ba3643faee91f6cea35824698/merged",
"UpperDir": "/var/docker/overlay2/9911ec636156cdd8d1c9e3256587c914ea875b9ba3643faee91f6cea35824698/diff",
"WorkDir": "/var/docker/overlay2/9911ec636156cdd8d1c9e3256587c914ea875b9ba3643faee91f6cea35824698/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "bind",
"Source": "/xxxx/yyyy/mongodb/shared",
"Destination": "/data/shared",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "133f77e6d00f",
"Domainname": "",
"User": "root",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"27017/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"no_proxy=*.local, 169.254/16",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"http_proxy=http://proxy:xxxx",
"https_proxy=http://proxy:xxxx",
"JETTY_VERSION=9.2.23.v20171218"
],
"Cmd": null,
"Image": "masked-dockerimage-host:tag",
"Volumes": {
"/data/shared": {}
},
"WorkingDir": "",
"Entrypoint": [
"/data/shared/entrypoint.sh"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "3f7d146d57d71823a7c0af805f0e50fed6081339e883674abbad2127980cd1a5",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "xxxxx",
"com.docker.compose.service": "xxxxx",
"com.docker.compose.version": "1.17.1"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "9caa7a0e2e1da9e64d40005b530af40f6d7e8877bec9eea06c48e209c80079e6",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"27017/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "27017"
}
]
},
"SandboxKey": "/var/run/docker/netns/9caa7a0e2e1d",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"my-custom-network": {
"IPAMConfig": {},
"Links": null,
"Aliases": [
"133f77e6d00f"
],
"NetworkID": "26d427d5e938a54546e0e5661868fec112e3a9ce5c3779eb4079ce48c78638ce",
"EndpointID": "7d60c0be429f89125428a1f4c2f6dc7191faad2c5c643ff4bc2413974374bf6b",
"Gateway": "172.18.0.1",
"IPAddress": "172.18.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:12:00:02",
"DriverOpts": null
}
}
}
}
]
Not in VPN but I do have a VPN client in the MAC.
Any pointers on why it works for sometime and then not? Networking Issue?