Layers go missing between RUN statements

I’m trying to install msys2 using the Chocolatey package manager. My install command appears to complete successfully, but attempts to use installed files in subsequent RUN statements shows that changes from the install RUN statement are no longer present.

I’ve been able to reproduce this with a pretty small dockerfile:

FROM microsoft/windowsservercore

ENV chocolateyUseWindowsCompression=false

RUN @powershell \
      -NoProfile \
      -ExecutionPolicy Bypass \
      -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" \
 && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin" \
 && md C:\tools \
 && type nul > C:\tools\checkpoint1

RUN type nul > C:\tools\checkpoint2 \
 && choco install msys2 --package-parameters="/NoUpdate" --confirm \
 && type nul > C:\tools\checkpoint3 \
 && dir C:\tools

RUN dir C:\tools

The first RUN statement installs Chocolatey, adds choco to the PATH, makes a directory, and adds a checkpoint file to that directory.

The second RUN statement makes a second checkpoint file, uses choco to install msys2, makes a third checkpoint file, and then prints the contents of our directory.

The third RUN statement repeats the print of our directory.

The sane expectation is for the output of dir in the second and third RUN statements to be the same.
The actual result is that all files created in the second RUN statement are absent.

Output follows:

Step 1/5 : FROM microsoft/windowsservercore
 ---> ad6116672030
Step 2/5 : ENV chocolateyUseWindowsCompression=false
 ---> Using cache
 ---> 33b4c1cfe12f
Step 3/5 : RUN @powershell       -NoProfile       -ExecutionPolicy Bypass       -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"  && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"  && md C:\tools  && type nul > C:\tools\checkpoint1
 ---> Running in a027ec3b57c4
Getting latest version of the Chocolatey package for download.
Getting Chocolatey from https://chocolatey.org/api/v2/package/chocolatey/0.10.10.
Downloading 7-Zip commandline tool prior to extraction.
Extracting C:\Users\ContainerAdministrator\AppData\Local\Temp\chocolatey\chocInstall\chocolatey.zip to C:\Users\ContainerAdministrator\AppData\Local\Temp\chocolatey\chocInstall...
Installing chocolatey on this machine
Creating ChocolateyInstall as an environment variable (targeting 'Machine')
  Setting ChocolateyInstall to 'C:\ProgramData\chocolatey'
WARNING: It's very likely you will need to close and reopen your shell
  before you can use choco.
Restricting write permissions to Administrators
We are setting up the Chocolatey package repository.
The packages themselves go to 'C:\ProgramData\chocolatey\lib'
  (i.e. C:\ProgramData\chocolatey\lib\yourPackageName).
A shim file for the command line goes to 'C:\ProgramData\chocolatey\bin'
  and points to an executable in 'C:\ProgramData\chocolatey\lib\yourPackageName'.
Creating Chocolatey folders if they do not already exist.
WARNING: You can safely ignore errors related to missing log files when
  upgrading from a version of Chocolatey less than 0.9.9.
  'Batch file could not be found' is also safe to ignore.
  'The system cannot find the file specified' - also safe.
chocolatey.nupkg file not installed in lib.
 Attempting to locate it from bootstrapper.
PATH environment variable does not have C:\ProgramData\chocolatey\bin in it. Adding...
WARNING: Not setting tab completion: Profile file does not exist at
'C:\Users\ContainerAdministrator\Documents\WindowsPowerShell\Microsoft.PowerShe
ll_profile.ps1'.
Chocolatey (choco.exe) is now ready.
You can call choco from anywhere, command line or powershell by typing choco.
Run choco /? for a list of functions.
You may need to shut down and restart powershell and/or consoles
 first prior to using choco.
Ensuring chocolatey commands are on the path
Ensuring chocolatey.nupkg is in the lib folder
Removing intermediate container a027ec3b57c4
 ---> 5227d76fecc3
Step 4/5 : RUN type nul > C:\tools\checkpoint2  && choco install msys2 --package-parameters="/NoUpdate" --confirm  && type nul > C:\tools\checkpoint3  && dir C:\tools
 ---> Running in e23b859a6426
Chocolatey v0.10.10
Installing the following packages:
msys2
By installing you accept licenses for the packages.
Progress: Downloading msys2 20161025.0.0... 100%
Progress: Downloading chocolatey-core.extension 1.3.3... 100%
chocolatey-core.extension v1.3.3 [Approved]
chocolatey-core.extension package files install completed. Performing other installation steps.
 Installed/updated chocolatey-core extensions.
 The install of chocolatey-core.extension was successful.
  Software installed to 'C:\ProgramData\chocolatey\extensions\chocolatey-core'
msys2 v20161025.0.0 [Approved]
msys2 package files install completed. Performing other installation steps.
Installing to: C:\tools\msys64
Extracting 64-bit C:\ProgramData\chocolatey\lib\msys2\tools\msys2-base-x86_64-20161025.tar.xz to C:\tools\msys64...
C:\tools\msys64
Extracting C:\tools\msys64\msys2-base-x86_64-20161025.tar to C:\tools\msys64...
C:\tools\msys64
Starting initialization via msys2_shell.cmd
PATH environment variable does not have C:\tools\msys64 in it. Adding...
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).
 The install of msys2 was successful.
  Software installed to 'C:\tools\msys64'
Chocolatey installed 2/2 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
 Volume in drive C has no label.
 Volume Serial Number is EC03-67FA
 Directory of C:\tools
04/12/2018  03:04 PM    <DIR>          .
04/12/2018  03:04 PM    <DIR>          ..
04/12/2018  03:04 PM                 0 checkpoint1
04/12/2018  03:04 PM                 0 checkpoint2
04/12/2018  03:04 PM                 0 checkpoint3
04/12/2018  03:04 PM    <DIR>          msys64
               3 File(s)              0 bytes
               3 Dir(s)  135,626,526,720 bytes free
Removing intermediate container e23b859a6426
 ---> 7ea4e6a0433c
Step 5/5 : RUN dir C:\tools
 ---> Running in be3bde0c28e2
 Volume in drive C has no label.
 Volume Serial Number is EC03-67FA
 Directory of C:\tools
04/12/2018  03:04 PM    <DIR>          .
04/12/2018  03:04 PM    <DIR>          ..
04/12/2018  03:04 PM                 0 checkpoint1
               1 File(s)              0 bytes
               2 Dir(s)  136,087,236,608 bytes free
Removing intermediate container be3bde0c28e2
 ---> 9302aac014ec
Successfully built 9302aac014ec

I’ve spent several days looking for existing posts or bug reports for this problem, but so far haven’t had any luck. Does anyone out there know what might be going on here? Any help is desperately appreciated!

Resolved in Issue: https://github.com/docker/for-win/issues/1947
That issue is marked as “closed” (not “resolved”) because it’s caused by a bug in an upstream dependency.

Bugfix PR: https://github.com/Microsoft/hcsshim/pull/165

No information about upstream release schedule at this time.