Mutable image with docker run

I am using docker run (client and server version 1.10.3) with a devicemapper storage driver in direct-lvm mode. I was surprised that a change I made inside the container - creating some files - appears to have been made in the image itself. If I exit the container and use docker run again to start a new container, my changes are still there in the new container! This is not how I remember docker working when I last used it. How can I reset the image to what it was before, to undo my changes?

I think I made a mistake. I assumed that because rpm did not know about a file, that it must have been created by me running a script. However, when I actually tried manually creating a file, it did not persist after I exited the container, as expected. Perhaps that file that rpm did not know about was created by an RPM post-script.