High CPU usage for Django application

I use docker and compose for my day to day devvelopment work, mostly I’m working on a Django application. I currently use docker-machine on VMWare Fusion, which works well.

Expected behavior

Virtually no CPU usage when the application is idle. And low usage when browsing around in the application. (using the vmware-fusion setup with docker machine as a reference point.)

Actual behavior

The CPU usage for com.docker.driver.amd64-linux sits at around 100% and the osxfs process at around 50% while the application is idle. Disabling auto-reload takes care of this problem, but that is just a workaround as the autoreload feature is really useful. With autoreload disabled the CPU usage for both processes drops down to around 0%, which is the expected behaviour.

When browsing around on the Django application the CPU usage for the com.docker.driver.amd64-linux jumps to around 50%, the osxfs process has negligible CPU usage.

Information

$ pinata diagnose -u MA-2065 [1]
OS X: version 10.11.4 (build: 15E65)
Docker.app: version v1.11.0-beta8.2
Running diagnostic tests:
[OK] docker-cli
[OK] Moby booted
[OK] driver.amd64-linux
[OK] vmnetd
[OK] osxfs
[OK] db
[OK] slirp
[OK] menubar
[OK] environment
[OK] Docker
[OK] VT-x
Docker logs are being collected into /tmp/20160422-101846.tar.gz
Most specific failure is: No error was detected
Your unique id is: 08EE91BC-5DDA-496A-99C4-3446D2042370
Please quote this in all correspondence.

Edit: I did some further digging and it looks like the network is part of the problem, even a simple Go application running inside Docker spikes the CPU to around 50% when responding to a HTTP query, and the response is really slow, on the order of seconds. In the vmware setup it responds in <10ms.

Steps to reproduce the behavior

Run a Django 1.5 application inside a centos7 container.

Hi,

Could you please provide detailed information (a base Dockerfile? a Hub image?) about your use case and any steps necessary to provoke the behavior? From reviewing the Django autoreloader source https://github.com/django/django/blob/master/django/utils/autoreload.py#L57=L66, it looks like the package pyinotify is tried and, if unavailable, stat polling on every file with a 1s interval is used.

Regarding your Go test, we would be very interested to see what code you used to test and the steps necessary to reproduce your response time observation. Could you post that somewhere or paste it into this thread?

We are actively working on a number of performance improvements related to file sharing. We will investigate the behavior of Machine + VMWare Fusion to ensure that Docker for Mac can meet or exceed that performance.

Thanks for participating in the Docker for Mac Beta!

David

1 Like

Hi David,

Thanks for the reply, unfortunately the source for both apps is proprietary. However, I can easily reproduce the problem with a far simpler use case.

package main

import (
	"io"
	"net/http"
	"log"
)

// hello world, the web server
func HelloServer(w http.ResponseWriter, req *http.Request) {
	io.WriteString(w, "hello, world!\n")
}

func main() {
	http.HandleFunc("/hello", HelloServer)
	log.Fatal(http.ListenAndServe(":12345", nil))
}

$ docker run -vpwd:/go -p 12345:12345 golang:1.6-alpine go run slow.go
$ time curl docker.local:12345/hello hello, world! curl docker.local:12345/hello 0.00s user 0.01s system 0% cpu 5.558 total

The golang:1.6-alpine image was already pulled in by the way.

As you can see, the hello world example from the go docs runs in 5s.

As for the Django app, I’ve installed py-inotify, and that seems to have solve the polling issue.

I believe the slow HTTP request is due to a DNS timeout related to the use of mDNS for docker.local. See Slow IP resolution of docker container for more discussion about the timeout. If you manually resolve docker.local, does the request latency drop to something more reasonable?

I’m glad py-inotify has solved your polling issue. We are continuing to make file sharing performance improvements.

After a reboot docker.local doesn’t resolve anymore, I’m guessing that it is a conflict with fusion, perhaps in addition to the timeouts you mention. I’ll keep trying and report back when I know more.

In the mean time I would like to thank you and the other Docker people for their hard work, Docker is a great tool and I’ve been using it for day-to-day development for over a year now. Thanks!

Ok, so I’ve got it working again. So I tried curling the IP instead of docker.local and that works like it’s supposed to.

That basically leaves me with one problem, the responses are still slower then what machine on top of fusion does. These are some timings from the login page of the aforementioned Django app:

curl http://192.168.64.3:8001/login > /dev/null 0.00s user 0.00s system 1% cpu 0.738 total
curl http://172.16.75.128:8001/login/ > /dev/null 0.00s user 0.00s system 3% cpu 0.240 total

The former one is docker for mac, the latter one is machine on top of fusion. As you can see it is quite a difference. The thing that sticks out to me is the fact that CPU utilisation jumps to 100% on docker for mac (com.docker.driver.amd64-linux), while the fusion process uses virtually 0% CPU.

I’d like to contribute by sharing my experience: Constantly around 50% CPU usage with no activity with Django development server and Docker for Mac beta.
I checked django docs for development server and it does indeed state directly that development server uses a slow polling method that can be fixed by pip install pyinotify and restarting the dev server. Worked for me.

However, I did not see high CPU usage using VirtualBox with the exact same setup so this is probably a good place to optimize. Or add it as a point to a ‘known issue’ page which could give a hint to https://docs.djangoproject.com/en/1.9/ref/django-admin/#django-admin-runserver for Django and a similar link for nodejs.

1 Like

this definitely worked and brought the CPU usage down to an average of 3% for me