What is the right nomenclature for volume_from in compose V2?

[devmtl]

Hey guys,

I have a data-container (data-html) that have static site files here:
$ pwd
/data/extracted-html
$ ls -la

drwxr-xr-x 5 root root 4096 Apr 21 13:56 .
drwxr-xr-x 3 root root 4096 Apr 21 13:56 ..
drwxr-xr-x 2 root root 4096 Apr 21 13:56 css
drwxr-xr-x 3 root root 4096 Apr 21 13:56 images
-rwxr-xr-x 1 root root 2794 Apr 21 13:56 index.html
drwxr-xr-x 2 root root 4096 Apr 21 13:56 js

Life is good.

Now I my main compose V2 file I have a nginx container that serves static content. here is my compose file: https://gist.github.com/pascalandy/215d4e99895c71cb541ca7ab7c7ef63c

I’m confused about how to use volumes_from:

I feel there should be 3 values here:

• container source (data-html)
• path from the container’s source (/data/extracted-html)
• path from the container’s destination (/usr/share/nginx/html)

The docs at https://docs.docker.com/compose/compose-file/#volumes-from seems too simplistic to help me understand. Any cues?

Pascal
twitter.com/_pascalandy

[nathanleclaire]

You don’t need to use volumes_from if using version 2. You can create a volume in volumes section and have it shared between containers.

$ cat docker-compose.yml
version: '2'

volumes:
  quux:
    driver: local

services:
  foo:
    image: alpine
    volumes:
      - quux:/some/path
    command: sh -c 'echo hello >/some/path/greeting'

  bar:
    image: alpine
    volumes:
      - quux:/another/path
    command: sh -c 'cat /another/path/greeting'
$ docker-compose run foo
$ docker-compose run bar
hello

[devmtl]

Thanks for this @nathanleclaire!

So if I understand well, we can define

volumes:
quux:
driver: local

in each of our compose file right?

Typically, in our apps and our data container. I’ll try this ASAP

Cheers!

Pascal
twitter.com/_pascalandy

[devmtl]

Hey Nathan,
I’m sure I’m close but not quite yet.

1. I have a data container (named data-html) where I wget static file for a webpage. Here is my data:

root@8758382265f1:/data/get-html# ls
2016-04-21_17h25.md  css  images  index.html  js

2. I’m using the official nginx (named ngx-html) image to serve those files over /usr/share/nginx/html/

The challenge is to have nginx gets the files in the data-html.

Here are my two compose files:

1. for data-html

version: "2"
volumes:
  magic-vol:
    driver: local
services:
  data-html:
    container_name: data-html
    volumes:
      - magic-vol:/data/get-html
    build: .
    networks:
      - front
networks:
  front:
    external:
      name: front-ntwk

2. for nginx

version: "2"
volumes:
  magic-vol:
    driver: local
services:
  ngx-html:
    container_name: html.g99999101_136
    env_file:
        - ./ngx-static.env
    # The build: uses the original DockerFile from nginx team.
	  build: ./static
    volumes:
      - magic-vol:/data/get-html/:/usr/share/nginx/html/
    restart: always
    networks:
      - front
networks:
  front:
    external:
      name: front-ntwk

There is probably an issue with this:

volumes:
      - magic-vol:/data/get-html/:/usr/share/nginx/html/

This setup also crash the original index.html provided by the nginx image :-/

Thanks in advance!
Pascal twitter.com/_pascalandy

[nathanleclaire]

Just at a quick glance, you don’t need a data container if you are using named volumes, so I highly suggest you skip it. Also, if resources are meant to be shared/interact, they should be in the same Compose file, not multiple ones.

[devmtl]

You’re correct! In this case the data container download about 2 Mo so I’m ok to use it in the same compose file, but other data-container are much bigger. That why I experiment using two compose files.

[nathanleclaire]

Also, you don’t use the bindmount notation if the volume is named. name:/foo:/bar is invalid (it doesn’t make sense – named volumes are meant to be shared between containers but not from host <=> container).

[nathanleclaire]

Why does the size of the container matter for whether or not it’s in the same Compose file?

[devmtl]

I understand the logic:

Also, you don’t use the bindmount notation if the volume is named. name:/foo:/bar is invalid (it doesn’t make sense – named volumes are meant to be shared between containers but not from host <=> container).

Now how do I indicate to nginx to read data (it expects at /usr/share/nginx/html/) from the magic-vol:/data/get-html

version: "2"

volumes:
  magic-vol:
    driver: local
    
services:
  data-html:
    container_name: data-html
    volumes:
      - magic-vol:/data/get-html
    build: ./data-html
    networks:
      - spring
    restart: always
    
  ngx-html:
    container_name: html.g99999101_136
    env_file:
        - ./ngx-static.env
    build: ./static
    volumes:
      - magic-vol:/data/get-html/
    restart: always
    networks:
      - spring
      
networks:
  spring:
    external:
      name: spring-ntwk

[devmtl]

@nathanleclaire

Why does the size of the container matter for whether or not it’s in the same Compose file?

Delay. I sometime download 100-200 MO in the data-ctn. I know the best thing would be to have a CICD that builds automatically images, but I like to flexibility to build from the Dockerfile.

Soon my stack will mature but now, a flexible data-ctn is my swiss knife

Make sense?

[nathanleclaire]

nginx:
  volumes:
    - magic-vol:/usr/share/nginx/html

[nathanleclaire]

@devmtl

Delay. I sometime download 100-200 MO in the data-ctn. I know the best thing would be to have a CICD that builds automatically images, but I like to flexibility to build from the Dockerfile.

Soon my stack will mature but now, a flexible data-ctn is my swiss knife

Make sense?

I have no idea what data you’re loading up, but you’re probably much better off just having some sort of service which is meant to be docker-compose runed that “seeds” the data (I’m guessing by slurping it down from an S3 bucket, the web, etc.). That way you can load it up optionally, but at any rate, I my opinion is that you should move away from baking data directly into a container image. It conflates otherwise separate concerns.

[devmtl]

First, I want to confirm that your info is correct. Thank you very much for this! Now it makes sense.
volumes:
- magic-vol:/usr/share/nginx/html

Docs question: Is there a bunch of best practice - real world examples of Docker stacks somewhere? I see https://github.com/docker/docker/blob/master/docs/userguide/containers/dockervolumes.md but nothing about dkr-cmp v2 and volume drivers.

About:

@nathanleclaire

just having some sort of service which is meant to be docker-compose ran that “seeds” the data (I’m guessing by slurping it down from an S3 bucket, the web, etc.).

This is exactly what I’m doing (or trying to do). I decided to put both containers into the same compose v2. Previously I was messing around and thought it would be easier the debug my stack that way.

I’m curious to understand why you are affirming it (“seeds” the data) is not the case. nginx container serve files form the data-html container. My POV here, is that I don’t want to put the static files along my compose file project. I want this to be pulling at launch (from git or s3 bucket). Isn’t it separating the concerns enough?

Thanks for everything you put out there in the world. I appreciate and make those xxx+ hours spend on this worth it!

Cheers!
twitter.com/_pascalandy

[devmtl]

Hi @nathanleclaire ,

Can you refer me to docs about Volume drivers for compose v2?

volumes:
  DrivePsqlBkp:
    driver: local

I need to share DrivePsqlBkp between containers that are not launched in the same docker-compose.yml (v2). What are the other options than local?

---

Or is this the way to go?

volumes:
  DrivePsqlBkp:
    external: true

So now there is no driver definition … I’m confused.

Thanks again!
Pascal

[nathanleclaire]

Compose file version 3 reference

If you need the containers to services resources (i.e. volumes), put them in the same Compose file.