Docker Slower to Copy Files and Run Compiler in Server 2019 than Windows 10

Expected behavior

Docker container runs at the same speed between Windows 10 and Windows Server 2019 when copying files to\from a container or compiling source code

Actual behavior

Docker container is signficantly slower to copy files or run the compiler in a container

Information

Hi

We are running a set of servers with Docker and have found that our servers with Windows Server 2019 with Mirantis Container Runtime run the same Docker container slower than those running Windows 10 with Docker Desktop.
The job is triggered by Jenkins and carries out the following key steps:

  1. copies source code from a bind mount to inside the container
  2. then runs a compiler on the source code on the files in the container
  3. then copies the build results from the container to the bind mount again.

The file copy takes

  • ~ 4 minutes on Windows Server 2019
  • 40s on Windows 10

The compiler takes

  • 55 seconds on Windows Server 2019
  • 40s on Windows 10

I’ve also tried swapping the Docker install on each server as below
Mirantis Container Runti* me on Windows 10

  • Docker Desktop on Server 2019
    and the results are the same.
    Windows 10 is significantly faster to copy the files and run the compiler container than Windows Server 2019.
    Both servers are running on VMs on the same VM server.

Does anyone have an idea for why this would be?

Details of the servers below:
Windows Server 2019

  • 10.0.17763.2114
  • Intel Xeon Gold 5281R @ 2.10 GHz 2 sockets, 16 cores
  • 16GB RAM

Mirantis Container Runtime
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker Application (Docker Inc., v0.8.0)
cluster: Manage Mirantis Container Cloud clusters (Mirantis Inc., v1.9.0)
registry: Manage Docker registries (Docker Inc., 0.1.0)

Server:
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 3
Server Version: 20.10.6
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics internal l2bridge l2tunnel nat null overlay private transparent
Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Standard Version 1809 (OS Build 17763.2114)
OSType: windows
Architecture: x86_64
CPUs: 16
Total Memory: 16GiB
Name: DEAEC-EE-DODev2
ID: VLTY:FNY4:K4LG:62D2:UNIU:AR6W:CKSD:VIDG:DU35:XMX4:QRP5:NQ2X
Docker Root Dir: C:\ProgramData\docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

Windows 10

  • 10.0.19043.1165
  • Intel Xeon Gold 5281R @ 2.10 GHz 1 sockets, 4 cores
  • 16GB RAM

Docker Desktop
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-beta.6)
scan: Docker Scan (Docker Inc., v0.8.0)

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1
Server Version: 20.10.7
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics internal l2bridge l2tunnel nat null overlay private transparent
Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 19043 (19041.1.amd64fre.vb_release.191206-1406)
Operating System: Windows 10 Pro Version 2009 (OS Build 19043.1165)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 16GiB
Name: AEC-EE-DODev3
ID: MW3G:CEMR:FODF:RQ23:3GS2:ZVAJ:TBZX:757D:IDUW:ISRX:BJCP:7DAZ
Docker Root Dir: C:\ProgramData\docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

For completion I found a solution that works for me, but doesn’t make a lot of sense

I tested running the container with HyperV isaltion mode on Windows Server instead of the standard process isolation.
It’s almost as fast as it is on Windows 10 using HyperV

The file copy takes
• ~ 4 minutes on Windows Server 2019 with standard process isolation
• 60 s on Windows Server 2019 with HyperV isolation
• 40 s on Windows 10
The compiler takes
• 55 s on Windows Server 2019 with standard process isolation
• 37 s on Windows Server 2019 with HyperV isolation
• 40 s on Windows 10

I’ve repeated this several times and it’s the same. I even tried specifying process isolation on Windows Server with the Docker run argument and it’s as slow as without the argument.
I guess I have a solution for now, but I’m not clear on why this would be so.