Docker Temp Storage

We use a docker container to run our software application on a jetson device where the whole operating system is running from an external microSD. At the moment, images are stored on the microSD and they are accessible after power cycle.

However, we want to add a “secure mode” where images are NOT stored in anyway when power is removed. Its important that the images are lost when device is powered off and not by our software deleting them.

One of my engineers says we can simply store the images within the Docker container and this will lead to the images being lost when the docker is manually stopped or the device is rebooted.

My question is, bearing in mind the whole OS is running from the external microSD, if we store the images in the Docker Container, would it be theoretically possible for images to be recovered from the microSD after power cycle?

A containers filesystem consists of a virtual filesystem that uses the image layers and a copy-on-write-layer. If the image and its layers are lost, then a container based on them will not be able to work…

I get the feeling that your engineer thinks a container uses a full clone of the image as filesystem, but in fact it is not.

Thank you for your response.

So does that mean that files CAN be theoretically recovered even after power cycle?

Btw when I say “image” I mean photos and potentially video files

If docker and not docker swarm is used, and the container exists after a power cycle, then the state of the container filesystem will be available.

Though, it is not recommended to keep files to be persisted in the container filesystem. Instead it is recommend to either use bind-volues (probably what you aready do) or named volumes baked by a remote fileshare (cifs/nfs). A containers file system is deleted, when the container is deleted - there is no way to recover from the deletion. Are you aware of the container update proces, which requires the old container to be deleted and a new to be created based on the updated image - docker-compose does the same under the hood!

I strongy suggest to not store persistant data in a containe that is supposed to be ephemeral by nature.

As I understand it, in this case this is what @alexvizgard wants. Loosing that data.

@alexvizgard When a container stops it doesn’t mean it is automatically removed unless you create (run) the container with --rm, but I think it is not possible with docker-compose up only with docker-compose run and docker run. In case of power loss however, the containers can’t be properly stopped, but I guess data would be removed when Docker daemon starts again. It means it would be still possible to recover data by booting from an other system and copying data from the disk.

Depending on the amount of data, you can use tmpfs to save files into memory.

Maybe, but I can’t help you with that, you could encrypt the data with different keys every time the container starts (or based on system boots) and store the key in memory. That way it would be at least very hard if not impossible to recover the data without knowing the old key.

The last idea was just the result of my fantasy, I don’t know how it could be achieved.

2 Likes

Hi @rimelek, yes that’s exactly it.

If I understand correctly, the tmpfs would take up some in the memory which on our current jetson is 8GB. If we were to instead buy a 16GB memory version, then we’d have 8GB to play with which should be more than enough for our needs.

That option sounds like the easiest thing to do, but your idea about storing the encryption key in memory so that people cannot try and access the data on boot up before docker clears the old data is an interesting backup option.

Thank you v much for your input!