Docker Community Forums

Share and learn in the Docker community.

Docker container stuck restarting after power failure


(Salvettimarc) #1

Hello I have a container that is occasionally unable to restart after a power cut.

docker restart/stop containername or even docker rm -f containername just hangs forever without result or log output.

There is a restart unless-stopped policy set on the container.

However, after a manual reboot of the host everything goes back to normal, container just restart normally.

$ docker info
Containers: 27
 Running: 16
 Paused: 0
 Stopped: 11
Images: 161
Server Version: 1.12.0
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 539
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host null overlay
Kernel Version: 4.4.0-42-generic
Operating System: Ubuntu 16.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.58 GiB
Name: lucypos1
ID: 4SRL:7RCJ:UKCW:MO3J:DMYM:ELG6:TB44:H53Y:NJCC:HG3Q:ICIT:RMV5
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Labels:
 provider=generic

$ docker version
Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 22:00:43 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 22:11:10 2016
 OS/Arch:      linux/amd64

 $ uname -a
Linux marco-newlap 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

output of docker logs containername after 1st restart attempt (any further attempt don’t seem to produce any effect or log output)

2016-10-18 08:11:57,370 WARN received SIGTERM indicating exit request
2016-10-18 08:11:57,370 INFO waiting for cron, hhvm to die
2016-10-18 08:11:59,380 INFO stopped: hhvm (terminated by SIGTERM)
2016-10-18 08:11:59,381 INFO stopped: cron (terminated by SIGTERM)
Traceback (most recent call last):
File “/usr/bin/supervisord”, line 9, in
File “/usr/lib/python2.7/dist-packages/pkg_resources/init.py”, line 542, in load_entry_point
File “/usr/lib/python2.7/dist-packages/pkg_resources/init.py”, line 2569, in load_entry_point
File “/usr/lib/python2.7/dist-packages/pkg_resources/init.py”, line 2229, in load
File “/usr/lib/python2.7/dist-packages/pkg_resources/init.py”, line 2235, in resolve
File “/usr/lib/python2.7/dist-packages/supervisor/supervisord.py”, line 41, in
File “/usr/lib/python2.7/dist-packages/supervisor/options.py”, line 53, in
File “/usr/lib/python2.7/dist-packages/supervisor/xmlrpc.py”, line 12, in
File “/usr/lib/python2.7/dist-packages/supervisor/medusa/http_server.py”, line 24, in
File “/usr/lib/python2.7/dist-packages/supervisor/medusa/producers.py”, line 15, in
RuntimeError: unable to get file status from ‘/usr/lib/python2.7/asynchat.py’