Building a variant of an image with an additional large layer - any way to save space?

I have a docker image that describes an environment with a bunch of tools, compilers, etc. This is distributed to users using a private docker registry.

Under special circumstances I need to install an additional program in the docker image but it requires a lot of space so I don’t want it being distributed to all users everytime they pull the image.

Naive solution
Making a new image with Dockerfile:

FROM <base image>
<Install program>

but this would take up huge amounts of space as the base image changes over time.

Is there any way to save space without having to put large application as one of the first layers in my docker image ? My best guess at the moment is to distribute the application outside a docker image and mounting it into the container but that sort of ruins my docker setup.