Docker build creates layers by running containers

Hello,
I am creating RabbitMQ docker image to run on Windows. The dockerfile is as following:

# escape=`
FROM microsoft/windowsservercore:10.0.14393.1198

# install Erlang
RUN powershell -command `
iwr http://erlang.org/download/otp_win64_19.3.exe -OutFile c:\otp_win64_19.3.exe; `
c:\otp_win64_19.3.exe /S /D=C:\Erl8.3; `

# install RabbitMQ
RUN powershell -command `
iwr https://www.rabbitmq.com/releases/rabbitmq-server/v3.6.10/rabbitmq-server-windows-3.6.10.zip -OutFile c:\rabbitmq-server-windows-3.6.10.zip; `
Expand-Archive -Path c:\rabbitmq-server-windows-3.6.10.zip -DestinationPath C:\ -Force; `

# clean up files in separate layer due locks on files induced by RUN powershell -command
RUN powershell -command `
Remove-Item c:\otp_win64_19.3.exe; `
Remove-Item c:\rabbitmq-server-windows-3.6.10.zip

# setup environment variables
ENV ERLANG_HOME="C:\Erl8.3" RABBITMQ_SERVER="C:\rabbitmq-server-windows-3.6.10" RABBITMQ_LOG_BASE="C:\rabbitmq-server-windows-3.6.10\logs"

# publish ports
EXPOSE 4369 5671 5672 15672

# configure RabbitMQ
ADD RabbitMQ.config C:\Users\ContainerAdministrator\AppData\RabbitMQ\

# execute (mind that it runs as admin)
ENTRYPOINT ["%RABBITMQ_SERVER%\sbin\rabbitmq-server", "-detached"]
#ENTRYPOINT ["CMD"]

I see that on each RUN powershell command, the docker runs a container. As well, if the RUN command fails then the container remains there forever.

Is it an expected behavior to execute each RUN command in separate container (and leave garbage in case of errors)?