Docker Community Forums

Share and learn in the Docker community.

Volume or a non-volume directory?

Hi,

My application, running inside docker container, will download the files (size can range between 10-200MB) from cloud and then process them and then delete them. I don’t need to preserve these files. App will download and process the files in sequence. What is the recommended approach- volume and non-volume dir to store temp files in docker?

Non-volume data is stored in a layered overlay filesystem (in most distributions, this will be either an AUFS or DeviceMapper filesystem). The principle is the same in both cases (image source):

enter image description here

As already mentioned in comments, I can recommend reading the section “Understand images, containers and storage drivers” from the official documentation. This answer is just a short summary.

Each Docker image consists of multiple layers of filesystem images. For example, an Apache+PHP image might consist of (1) a generic Ubuntu base layer, (2) an additional layer with the Apache HTTP server installed and (3) another layer on top with PHP-FPM and configuration files (just an example).

When you start a new container from an image, a new per-container layer is added to the existing image layers. This layer will contain all changes that is written within the container itself (to non-volume directories).

Regarding your specific questions:

If you don’t mount any volumes, would it keep any/some/most/all of the container filesystem in RAM?

Nope, there’s nothing in RAM (besides the usual filesystem caches). It’s all in the overlay filesystem, which are mounted using AUFS, DeviceMapper or another storage driver.

Or does it write some amount of it to disk, meaning it would be just as slow as a volume in this case?

In general, filesystem access in volumes is more performant than in the overlay filesystem. After all, a volume (at least, a regular host-based volume, letting aside volume drivers that add network storage volumes) is simply a bind mount to a regular directory in the host filesystem, bypassing the layer filesystem entirely. The performance of volumes in comparison to layer filesystem is (among other topics) investigated in this paper:

AUFS introduces significant overhead which is not surprising since I/O is going through several layers, […]. Applications that are filesystem or disk intensive should bypass AUFS by using volumes. […] Although containers themselves have almost no overhead, Docker is not without performance gotchas. Docker volumes have noticeably better performance than files stored in AUFS.

1 Like

If you don’t care about the files after your container finished you don’t need a volume.

But if you want to cache the files and reuse the every time you restart the container a volume is a great way to share these files between different container runs.

1 Like

Thank you for the answers. I’ll give it a try with non-volume directory. As mentioned above my application is not I/O intensive and files does not need to be preserved after download.