How to avoid transferring everything as build context

How " or use variables inside the Dockerfile" will avoid sending all the binaries when whole current directory with all sub-directories is sent to the docker daemon regardless of what you put in the docker file?