Push to docker hub seems very slow

[rouilleur]

Hello,

I’m fairly new to docker and I was wondering what was the upload speed I could expect when pushing images to docker hub.
My pushes seems excessively slow. (it takes nearly 10 minutes to push a 16 MB image)

I found a lot of posts referring to slowness on docker for Mac, but I’m not using it.
I’m running docker inside a virtual box ubuntu VM on my windows 10 PC.
The VM is connected to internet through NAT adapter.
I have a good internet connection with displayed upload rate of 50 MB/s.
Pull speed seems totally correct.

My VM is also connected to a Host Only network, but this should not have any impact (I have multiples VM running a little swarm cluster).

Any help to improve the upload speed would be welcomed.

Regards,

Olivier

A bit more details:
docker version :

Client:
 Version:           18.09.5
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        e8ff056
 Built:             Thu Apr 11 04:43:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.5
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       e8ff056
  Built:            Thu Apr 11 04:10:53 2019
  OS/Arch:          linux/amd64
  Experimental:     false

I found a test script in another post:

Why is pushing an image so slow?

I am working with Docker since some weeks and I have a constant bad performance pushing images to the Docker registry. Something in the range of 1 MB/second can be considered fast. My dev machine has a fast uplink and working fast with other network services. The same slowness can be seen while pulling images. Is this a general slowness of the Docker registry? -aj

here is the output :

*************
CF delivery:
*************
18:48:27.600801 * Couldn't find host dseasb33srnrn.cloudfront.net in the .netrc file; using defaults
18:48:27.631271 *   Trying 143.204.222.95...
18:48:27.631334 * TCP_NODELAY set
18:48:27.636501 * Connected to dseasb33srnrn.cloudfront.net (143.204.222.95) port 443 (#0)
18:48:27.637712 * ALPN, offering h2
18:48:27.637841 * ALPN, offering http/1.1
18:48:27.669842 * successfully set certificate verify locations:
18:48:27.670624 *   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
18:48:27.671529 } [5 bytes data]
18:48:27.672901 * TLSv1.2 (OUT), TLS handshake, Client hello (1):
18:48:27.673786 } [226 bytes data]
18:48:27.685781 * TLSv1.2 (IN), TLS handshake, Server hello (2):
18:48:27.686236 { [76 bytes data]
18:48:27.686499 * TLSv1.2 (IN), TLS handshake, Certificate (11):
18:48:27.686646 { [4015 bytes data]
18:48:27.687720 * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
18:48:27.688003 { [333 bytes data]
18:48:27.688513 * TLSv1.2 (IN), TLS handshake, Server finished (14):
18:48:27.688661 { [4 bytes data]
18:48:27.689156 * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
18:48:27.689282 } [70 bytes data]
18:48:27.689544 * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
18:48:27.689709 } [1 bytes data]
18:48:27.690073 * TLSv1.2 (OUT), TLS handshake, Finished (20):
18:48:27.690216 } [16 bytes data]
18:48:27.695128 * TLSv1.2 (IN), TLS handshake, Finished (20):
18:48:27.695796 { [16 bytes data]
18:48:27.696321 * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
18:48:27.696543 * ALPN, server accepted to use http/1.1
18:48:27.696798 * Server certificate:
18:48:27.697070 *  subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.cloudfront.net
18:48:27.697285 *  start date: Oct  8 00:00:00 2018 GMT
18:48:27.697509 *  expire date: Oct  9 12:00:00 2019 GMT
18:48:27.697760 *  subjectAltName: host "dseasb33srnrn.cloudfront.net" matched cert's "*.cloudfront.net"
18:48:27.698025 *  issuer: C=US; O=DigiCert Inc; CN=DigiCert Global CA G2
18:48:27.698236 *  SSL certificate verify ok.
18:48:27.698796 } [5 bytes data]
18:48:27.699292 > GET /registry-v2/docker/registry/v2/blobs/sha256/61/614879c30ef309d95bb9f31ed3c9f24e83a79748f702cdc33841f6a9a133bedd/data?Expires=1557334167&Signature=TTmEJvu1YDs-~ZxLWPXateDF35xVj1CgJ997BJnDVBNeGBNShd0ks1oNRfzwtoogd9R1wTZA~19WrWmWE9BMrtMvmezazVwONa9nKKmErpl5CHi806unrOypSrjpcZJ6jVCwilBQGwSbJOhpjpgbqO0FRKvc9JwWW~6W1fk920E_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q HTTP/1.1
18:48:27.699292 > Host: dseasb33srnrn.cloudfront.net
18:48:27.699292 > User-Agent: curl/7.58.0
18:48:27.699292 > Accept: */*
18:48:27.699292 > 
18:48:28.126519 { [5 bytes data]
18:48:28.128259 < HTTP/1.1 200 OK
18:48:28.128463 < Content-Type: application/octet-stream
18:48:28.128666 < Content-Length: 188386234
18:48:28.128816 < Connection: keep-alive
18:48:28.128994 < Date: Wed, 08 May 2019 16:48:28 GMT
18:48:28.130094 < Last-Modified: Fri, 16 Oct 2015 08:50:59 GMT
18:48:28.130222 < ETag: "cc0e2b5dabd04d520591ac6e3a20262d"
18:48:28.130347 < x-amz-storage-class: INTELLIGENT_TIERING
18:48:28.130472 < x-amz-version-id: g.eNC7ZR9SrqL0SBFXT0jBbkAQfpdqmA
18:48:28.130612 < Accept-Ranges: bytes
18:48:28.130737 < Server: AmazonS3
18:48:28.130863 < X-Cache: Miss from cloudfront
18:48:28.131017 < Via: 1.1 9d82b8af3ace385f51687efdc41b21d1.cloudfront.net (CloudFront)
18:48:28.131175 < X-Amz-Cf-Id: wFJH1seJEx8QcS7Dd9Lkzpe7UXgP4_OjYvyxoukSWmmudk6OaMf9-Q==
18:48:28.131301 < 
18:48:28.131424 { [15842 bytes data]
18:48:32.051808 * Connection #0 to host dseasb33srnrn.cloudfront.net left intact

real	0m4,474s
user	0m0,541s
sys	0m2,183s
*************
Resolving:
*************

; <<>> DiG 9.11.3-1ubuntu1.7-Ubuntu <<>> resolver-identity.cloudfront.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60256
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;resolver-identity.cloudfront.net. IN	A

;; ANSWER SECTION:
resolver-identity.cloudfront.net. 1800 IN A	213.228.60.144

;; Query time: 12 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Wed May 08 18:48:31 CEST 2019
;; MSG SIZE  rcvd: 77

*************
RAW S3:
*************
18:48:31.115556 * Couldn't find host kamilc-us-west-1.s3.amazonaws.com in the .netrc file; using defaults
18:48:31.152200 *   Trying 52.216.98.99...
18:48:31.152357 * TCP_NODELAY set
18:48:31.239579 * Connected to kamilc-us-west-1.s3.amazonaws.com (52.216.98.99) port 80 (#0)
18:48:31.239837 > GET /test200m HTTP/1.1
18:48:31.239837 > Host: kamilc-us-west-1.s3.amazonaws.com
18:48:31.239837 > User-Agent: curl/7.58.0
18:48:31.239837 > Accept: */*
18:48:31.239837 > 
18:48:31.328096 < HTTP/1.1 404 Not Found
18:48:31.328312 < x-amz-request-id: E6EACDEE1D826DE6
18:48:31.328443 < x-amz-id-2: Y2JAuSJ+YfOd4d6knfuG+e6kY4/v5QJ+seO178rhFsqkCwZoBDf3ujojRZM5mVYqjKm/S3QNP8Q=
18:48:31.328573 < Content-Type: application/xml
18:48:31.328717 < Transfer-Encoding: chunked
18:48:31.328846 < Date: Wed, 08 May 2019 16:48:31 GMT
18:48:31.328977 < Server: AmazonS3
18:48:31.329107 < 
18:48:31.329249 { [318 bytes data]
18:48:31.406589 * Connection #0 to host kamilc-us-west-1.s3.amazonaws.com left intact

real	0m0,310s
user	0m0,013s
sys	0m0,074s
*************
RAW HTTP:
*************
18:48:32.427154 * Couldn't find host mirrors.sonic.net in the .netrc file; using defaults
18:48:32.555539 *   Trying 157.131.0.16...
18:48:32.555689 * TCP_NODELAY set
18:48:32.703185 * Connected to mirrors.sonic.net (157.131.0.16) port 80 (#0)
18:48:32.703427 > GET /centos/7/isos/x86_64/CentOS-7.0-1406-x86_64-NetInstall.iso HTTP/1.1
18:48:32.703427 > Host: mirrors.sonic.net
18:48:32.703427 > User-Agent: curl/7.58.0
18:48:32.703427 > Accept: */*
18:48:32.703427 > 
18:48:32.850316 < HTTP/1.1 404 Not Found
18:48:32.850475 < Date: Wed, 08 May 2019 16:48:32 GMT
18:48:32.850602 < Server: Apache/2.4.6 (CentOS)
18:48:32.850756 < Content-Length: 338
18:48:32.850922 < Content-Type: text/html; charset=iso-8859-1
18:48:32.851048 < 
18:48:32.851172 { [338 bytes data]
18:48:32.851475 * Connection #0 to host mirrors.sonic.net left intact

real	0m0,444s
user	0m0,011s
sys	0m0,008s

[maelstromeous]

I’m also getting very slow push speeds, in the region of 800kb/s today.

[cdupont2]

I have this problem too, with Ubuntu and docker 19.03.0.

[haydigo (Hayko)]

Me too, why is this happening? Something todo with the new pricing?

[danleis]

docker push is slow again. sometimes it gets better but now its insanely slow. I am wondering whether it is because I am using the free package or they do the same with people who actually pay them?

[devitocodes]

Tumbleweeds… Did anyone ever get an answer to this because push speeds really suck.

[devitocodes]

For the record - we already upgraded to pro and it didn’t make any difference to the upload speed. That was just a hail mary anyhow as the documentation doesn’t say that there should be any difference in upload speeds - only number of pulls.