Docker Community Forums

Share and learn in the Docker community.

Docker hang using interactive ubuntu, 300% CPU


(Faye Salwin) #1

Expected behavior

typing in shell produces output
running docker ps displays the process state

Actual behavior

no IO to container
faye.salwin:~ $ docker run -it --rm ubuntu:16.04
root@8bc062febda7:/# mkdir /netapp
root@8bc062febda7:/# mkdir /3par
root@8bc062febda7:/# mkdir /netapp/a
root@8bc062febda7:/# mkdir /netapp/b
root@8bc062febda7:/# mkdir /netapp/c
root@8bc062febda7:/# mkdir netapp
And when stopping docker:
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

docker ps hangs.
faye.salwin:~ $ docker ps
and on stopping Docker:
An error occurred trying to connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.23/containers/json: EOF

piñata times out gathering information

Information

  • the output of:

    • pinata diagnose -u on OSX
      faye.salwin:/ $ pinata diagnose -u
      OS X: version 10.10.4 (build: 14E46)
      Docker.app: version v1.11.1-beta13
      Running diagnostic tests:
      [OK] Moby booted
      [OK] driver.amd64-linux
      [OK] vmnetd
      [OK] osxfs
      [OK] db
      [OK] slirp
      [OK] menubar
      [OK] environment
      [OK] Docker
      [OK] VT-x
      Error echo “00000003.0000f3a6” | nc -U /Users/faye.salwin/Library/Containers/com.docker.docker/Data/@connect > /tmp/20160526-113002/diagnostics.tar: timeout after 30.00s
      Docker logs are being collected into /tmp/20160526-113002.tar.gz
      Most specific failure is: No error was detected
      Your unique id is: 33180A72-851C-408D-96A3-E33B0DEF08A4
      Please quote this in all correspondence.
  • a reproducible case if this is a bug, Dockerfiles FTW
    you can see what I did, not sure if it is reproducible. Maybe related to long standing 200%cpu bug but there was no suspend or low battery event.

  • host distribution and version ( OSX 10.10.x, OSX 10.11.x, Windows, etc )
    faye.salwin:/ $ uname -a
    Darwin fayes-omac-2.opower.local 14.4.0 Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64 x86_64
    Yosemite 10.10.4

Sampling process 24853 for 3 seconds with 1 millisecond of run time between samples
Sampling completed, processing symbols…
Analysis of sampling com.docker.hyperkit (pid 24853) every 1 millisecond
Process: com.docker.hyperkit [24853]
Path: /Applications/Docker.app/Contents/MacOS/com.docker.hyperkit
Load Address: 0x10047e000
Identifier: com.docker.hyperkit
Version: 0
Code Type: X86-64
Parent Process: com.docker.driver.amd64-linux [24851]

Date/Time: 2016-05-26 11:27:29.953 AM -0700
OS Version: Mac OS X 10.10.4 (14E46)
Report Version: 7
Analysis Tool: /usr/bin/sample

Call graph:
2670 Thread_7101022 DispatchQueue_1: com.apple.main-thread (serial)
+ 2670 start (in libdyld.dylib) + 1 [0x7fff8d6ce5c9]
+ 2670 kevent (in libsystem_kernel.dylib) + 10 [0x7fff8e5c021a]
2670 Thread_7101029
+ 2670 ??? (in ) [0x7f82a2412a20]
+ 2670 caml_start_program (in com.docker.hyperkit) + 92 [0x1005996e8]
+ 2670 .L101 (in com.docker.hyperkit) + 21 [0x100529949]
+ 2670 .L108 (in com.docker.hyperkit) + 5 [0x1004e4974]
+ 2670 .L322 (in com.docker.hyperkit) + 42 [0x1004e2de6]
+ 2670 .L378 (in com.docker.hyperkit) + 31 [0x1004e3223]
+ 2670 __select (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf3fa]
2670 Thread_7101030
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 __select (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf3fa]
2670 Thread_7101031
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 worker_loop (in com.docker.hyperkit) + 123 [0x10057c90b]
+ 2670 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf136]
2670 Thread_7101032
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 callout_thread_func (in com.docker.hyperkit) + 183 [0x100481d71]
+ 2670 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf136]
2670 Thread_7101035
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 pci_vtnet_tx_thread.1149 (in com.docker.hyperkit) + 234 [0x1004a5ec4]
+ 2670 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf136]
2670 Thread_7101036
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 __select (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf3fa]
2670 Thread_7101037
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 blockif_thr (in com.docker.hyperkit) + 247 [0x10048d0df]
+ 2670 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf136]
2670 Thread_7101038
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 __select (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf3fa]
2670 Thread_7101039
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 __select (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf3fa]
2670 Thread_7101040 DispatchQueue_2: com.apple.libdispatch-manager (serial)
+ 2670 _dispatch_mgr_thread (in libdispatch.dylib) + 52 [0x7fff8be8aa6a]
+ 2670 kevent64 (in libsystem_kernel.dylib) + 10 [0x7fff8e5c0232]
2670 Thread_7101041
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 vcpu_thread (in com.docker.hyperkit) + 1168 [0x1004a2300]
+ 2670 xh_vm_run (in com.docker.hyperkit) + 663 [0x1004823e8]
+ 2664 vmx_run (in com.docker.hyperkit) + 354 [0x100484071]
+ ! 2664 hv_vcpu_run (in Hypervisor) + 16 [0x100a310a9]
+ 2 vmx_run (in com.docker.hyperkit) + 375 [0x100484086]
+ ! 2 vcpu_read_vmcs_id (in Hypervisor) + 31,50 [0x100a31997,0x100a319aa]
+ 2 vmx_run (in com.docker.hyperkit) + 8006 [0x100485e55]
+ ! 1 fls (in libsystem_platform.dylib) + 5 [0x7fff8322798d]
+ ! 1 vlapic_pending_intr (in com.docker.hyperkit) + 49 [0x100489ce4]
+ 2 vmx_run (in com.docker.hyperkit) + 7997,357 [0x100485e4c,0x100484074]
2670 Thread_7101048
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 vcpu_thread (in com.docker.hyperkit) + 1168 [0x1004a2300]
+ 2670 xh_vm_run (in com.docker.hyperkit) + 663 [0x1004823e8]
+ 2665 vmx_run (in com.docker.hyperkit) + 354 [0x100484071]
+ ! 2665 hv_vcpu_run (in Hypervisor) + 16 [0x100a310a9]
+ 3 vmx_run (in com.docker.hyperkit) + 7865,7721 [0x100485dc8,0x100485d38]
+ 2 vmx_run (in com.docker.hyperkit) + 375 [0x100484086]
+ 2 hv_vmx_vcpu_read_vmcs (in Hypervisor) + 0 [0x100a31620]
2670 Thread_7101053
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 vcpu_thread (in com.docker.hyperkit) + 1168 [0x1004a2300]
+ 2670 xh_vm_run (in com.docker.hyperkit) + 663 [0x1004823e8]
+ 2670 vmx_run (in com.docker.hyperkit) + 354 [0x100484071]
+ 2670 hv_vcpu_run (in Hypervisor) + 16 [0x100a310a9]
2670 Thread_7101054
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 vcpu_thread (in com.docker.hyperkit) + 1168 [0x1004a2300]
+ 2670 xh_vm_run (in com.docker.hyperkit) + 1600 [0x100482791]
+ 2670 __psynch_cvwait (in libsystem_kernel.dylib) + 10 [0x7fff8e5bf136]
2670 Thread_7101142
+ 2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
+ 2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
+ 2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
+ 2670 read (in libsystem_kernel.dylib) + 10 [0x7fff8e5c0682]
2670 Thread_7101143
2670 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
2670 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
2670 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
2670 read (in libsystem_kernel.dylib) + 10 [0x7fff8e5c0682]

Total number in stack (recursive counted multiple, when >=5):
14 _pthread_body (in libsystem_pthread.dylib) + 131 [0x7fff8d1d2268]
14 _pthread_start (in libsystem_pthread.dylib) + 176 [0x7fff8d1d21e5]
14 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff8d1d041d]
5 __psynch_cvwait (in libsystem_kernel.dylib) + 0 [0x7fff8e5bf12c]
5 __select (in libsystem_kernel.dylib) + 0 [0x7fff8e5bf3f0]

Sort by top of stack, same collapsed (when >= 5):
__psynch_cvwait (in libsystem_kernel.dylib) 13350
__select (in libsystem_kernel.dylib) 13350
hv_vcpu_run (in Hypervisor) 7999
read (in libsystem_kernel.dylib) 5340
kevent (in libsystem_kernel.dylib) 2670
kevent64 (in libsystem_kernel.dylib) 2670
vmx_run (in com.docker.hyperkit) 5

Sample analysis of process 24853 written to file /dev/stdout

26/05/2016 10:12:11.177 AM SubmitDiagInfo[1487]: Removed expired problem report: file:///Users/faye.salwin/Library/Logs/DiagnosticReports/com.docker.driver.amd64-linux_2016-04-25-195318_Fayes-OMac-2.crash
26/05/2016 10:12:11.178 AM SubmitDiagInfo[1487]: Removed expired problem report: file:///Users/faye.salwin/Library/Logs/DiagnosticReports/com.docker.driver.amd64-linux_2016-04-26-091759_Fayes-OMac-2.crash
26/05/2016 10:12:11.585 AM SubmitDiagInfo[1487]: Submitted problem report file:///Users/faye.salwin/Library/Logs/DiagnosticReports/com.docker.hyperkit_2016-05-25-173055_Fayes-OMac-2.crash
26/05/2016 10:12:11.586 AM SubmitDiagInfo[1487]: Submitted problem report file:///Users/faye.salwin/Library/Logs/DiagnosticReports/com.docker.hyperkit_2016-05-25-174416_Fayes-OMac-2.crash
26/05/2016 10:12:11.587 AM SubmitDiagInfo[1487]: Submitted problem report file:///Users/faye.salwin/Library/Logs/DiagnosticReports/com.docker.hyperkit_2016-05-25-181556_Fayes-OMac-2.crash

Steps to reproduce the behavior

not reliably reproduced.

Can’t upload the tar ball because this is a first post, I think.