Copies of layers appearing in push

When I build tag and push an image, the changes do not appear most of the time. There are copies of layers being used when the push step occurs.

The push refers to a repository [xxx]
ce2ffbd466d9: Preparing
1e5403e88a8d: Preparing
f0a480f1bb0d: Preparing
0aeb287b1ba9: Preparing
0aeb287b1ba9: Layer already exists
0aeb287b1ba9: Layer already exists
1e5403e88a8d: Pushed
1e5403e88a8d: Pushed
ce2ffbd466d9: Pushed
ce2ffbd466d9: Pushed
f0a480f1bb0d: Pushed
f0a480f1bb0d: Pushed

The commands used are

docker build --force-rm --no-cache --pull --rm=true -t xxx .
docker tag -f xxx:latest
docker push xxx:latest

When I remove older images with
docker ps -a | xargs docker rm -f
The changes will appear properly with the next build/tag/push, but this will have to happen after every build.
There are also multiple images (and copies of images) that are unable to be deleted due to

image has dependent child images
image is being used by running container 523bac53a29b

Is there a way to force build an image without using older layers?

EDIT: My solution was that I was working on a BuildSwarm and the dockerhost was not defaulting to the one I needed (export DOCKER_HOST=xxxxxxxxx)