Docker build creates layers by running containers

I am creating RabbitMQ docker image to run on Windows. The dockerfile is as following:

# escape=`
FROM microsoft/windowsservercore:10.0.14393.1198

# install Erlang
RUN powershell -command `
iwr -OutFile c:\otp_win64_19.3.exe; `
c:\otp_win64_19.3.exe /S /D=C:\Erl8.3; `

# install RabbitMQ
RUN powershell -command `
iwr -OutFile c:\; `
Expand-Archive -Path c:\ -DestinationPath C:\ -Force; `

# clean up files in separate layer due locks on files induced by RUN powershell -command
RUN powershell -command `
Remove-Item c:\otp_win64_19.3.exe; `
Remove-Item c:\

# setup environment variables
ENV ERLANG_HOME="C:\Erl8.3" RABBITMQ_SERVER="C:\rabbitmq-server-windows-3.6.10" RABBITMQ_LOG_BASE="C:\rabbitmq-server-windows-3.6.10\logs"

# publish ports
EXPOSE 4369 5671 5672 15672

# configure RabbitMQ
ADD RabbitMQ.config C:\Users\ContainerAdministrator\AppData\RabbitMQ\

# execute (mind that it runs as admin)
ENTRYPOINT ["%RABBITMQ_SERVER%\sbin\rabbitmq-server", "-detached"]

I see that on each RUN powershell command, the docker runs a container. As well, if the RUN command fails then the container remains there forever.

Is it an expected behavior to execute each RUN command in separate container (and leave garbage in case of errors)?