Data directory "/var/lib/postgresql/data" has wrong ownership... on Linux?

Hello there.

I’m running a docker-compose stack with the services below:

  • A Jupyter notebook service
  • A mlflow tracking server
  • A postgresql database to stock mlflow info.

When I try to run docker-compose build and up, I get the classical “data directory “/var/lib/postgresql/data” has wrong ownership” that happened on Windows, but I’m running it on Linux and I can’t find the cause. Please, find docker-compose.yml and respective Dockerfile’s on Dropbox - files.tar.gz - Simplify your life (I can’t upload files), and here you can see the full log:

postgres_1 | The files belonging to this database system will be owned by user “postgres”.
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale “en_US.utf8”.
postgres_1 | The default database encoding has accordingly been set to “UTF8”.
postgres_1 | The default text search configuration will be set to “english”.
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data … ok
postgres_1 | creating subdirectories … ok
postgres_1 | selecting dynamic shared memory implementation … posix
postgres_1 | selecting default max_connections … 20
postgres_1 | selecting default shared_buffers … 400kB
postgres_1 | selecting default time zone … Etc/UTC
postgres_1 | creating configuration files … ok
postgres_1 | running bootstrap script … 2020-07-28 19:47:30.936 UTC [73] FATAL: data directory “/var/lib/postgresql/data” has wrong ownership
postgres_1 | 2020-07-28 19:47:30.936 UTC [73] HINT: The server must be started by the user that owns the data directory.
postgres_1 | child process exited with exit code 1
postgres_1 | initdb: removing contents of data directory “/var/lib/postgresql/data”

Any help you could provide would be great. Thanks for your time and dedication.

You could use a named volume for postgres instead. Or is there a specific reason why you want to use the local file system for the data?

Hi, @tekki, and thanks for your response.

I would rather have it inside the project folder, I’m not sure if I could specify where to store the named volume.

By the way, I’ve tried it with a named volume this time and it’s not working either. But this time I just get

chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted

.

The docker-compose.yml file, in case I’m doing it wrong (if needed, I’ll change env variables into value, but I think the problem is in those last fields which I’ve already changed :slight_smile: ):

version: '3.7'
services:
  jupyter:
    user: root
    build:
      context: .
      dockerfile: ./docker/jupyter/Dockerfile
      target: ${JUPYTER_TARGET}
      args: 
        - MLFLOW_ARTIFACT_STORE=/${MLFLOW_ARTIFACT_STORE}
        - MLFLOW_VERSION=${MLFLOW_VERSION}
        - JUPYTER_BASE_IMAGE=${JUPYTER_BASE_IMAGE}
        - JUPYTER_BASE_VERSION=${JUPYTER_BASE_VERSION}
        - JUPYTER_USERNAME=${JUPYTER_USERNAME}
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${JUPYTER_TARGET}:${VERSION}
    ports:
      - "${JUPYTER_PORT}:${JUPYTER_PORT}"
    depends_on: 
      - mlflow
    environment: 
      MLFLOW_TRACKING_URI: ${MLFLOW_TRACKING_URI}
      JUPYTER_ENABLE_LAB: ${JUPYTER_ENABLE_LAB}
      NB_USER: ${JUPYTER_USERNAME}
      NB_UID: ${JUPYTER_UID}
      CHOWN_HOME: "yes"
      CHOWN_HOME_OPTS: '-R'
      CHOWN_EXTRA: ${JUPYTER_CHOWN_EXTRA}
      CHOWN_EXTRA_OPTS: '-R'
    volumes:
      - ./:/home/${JUPYTER_USERNAME}/work
      - ./${MLFLOW_ARTIFACT_STORE}:/${MLFLOW_ARTIFACT_STORE}
  mlflow:
    build:
      context: ./docker/mlflow
      args:
        - MLFLOW_VERSION=${MLFLOW_VERSION}
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${MLFLOW_IMAGE_NAME}:${VERSION}
    expose: 
      - "${MLFLOW_TRACKING_SERVER_PORT}"
    ports:
      - "${MLFLOW_TRACKING_SERVER_PORT}:${MLFLOW_TRACKING_SERVER_PORT}"
    depends_on: 
      - postgres
    environment: 
      MLFLOW_TRACKING_SERVER_HOST: ${MLFLOW_TRACKING_SERVER_HOST}
      MLFLOW_TRACKING_SERVER_PORT: ${MLFLOW_TRACKING_SERVER_PORT}
      MLFLOW_ARTIFACT_STORE: ${MLFLOW_ARTIFACT_STORE}
      MLFLOW_BACKEND_STORE: ${MLFLOW_BACKEND_STORE}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DATABASE: ${POSTGRES_DATABASE}
      POSTGRES_PORT: ${POSTGRES_PORT}
      WAIT_FOR_IT_TIMEOUT: ${WAIT_FOR_IT_TIMEOUT}
    volumes:
      - ./${MLFLOW_ARTIFACT_STORE}:/${MLFLOW_ARTIFACT_STORE}
  postgres:
    user: "${POSTGRES_UID}:${POSTGRES_GID}"
    build:
      context: ./docker/postgres
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${POSTGRES_IMAGE_NAME}:${VERSION}
    restart: always
    environment: 
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_USER: ${POSTGRES_USER}
    ports:
      - "${POSTGRES_PORT}:${POSTGRES_PORT}"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

By the way, I’m using Docker v19.03.12, build 48a66213fe on Linux Mint 19.03

… and this is the reason:

Hey, @meyay, thanks for the quick answer.

I’m running it through a Makefile, which contains:

POSTGRES_UID := $(shell id -u)
POSTGRES_GID := $(shell id -g)

Isn’t it correct? What should I do?

Is it mentioned in the description of the dockerhub page that the image actualy support/requires to set the user?

I would say so…

So according docs it should work for the debian based images. That’s a safe starting point for an analysis.

Since your docker-compose.yml does obfuscate all details about the image:tag, it is impossible to see which tag you actualy used. I guess this is the price you pay if you use make and envsubst to have dynamic templating :wink: The both statements from above look like valid Makefile syntax.

You might want to take a look at the entrypoint script for that particular version: https://github.com/docker-library/postgres, you can start your analysis in the _main() function.

And… I should change the title because I know what was happening #facepalm

I didn’t realize was using for these tests an old folder I had mapped from a NTFS HD (I run 1 HD for Linux, another for Windows and a shared NTFS one where this folder is mapped…).

Seems like the problem, even though I’m running this on Linux, comes from that last NTFS disk. I’ve tried with another location for the named volume on the Ext4 and it runs perfectly.

So thanks a lot for your dedication, and sorry for having you busy with this one :frowning:

don’t worry.

Btw. If you ever have the situation that you end up with variables in your compose template, you can trick envsubst in only using variables that actualy exist in env:

# param 1: docker-compose template file
# param 2: project name of the deployment
define deploy-compose
        envsubst "$$(env | sed -e 's/=.*//' -e 's/^/\$$/g')" < $(1) \
                |tee debug.yaml \
                |docker-compose --project-name $(2) -f - up -d  && rm debug.yaml;
endef

Maybe this might be usefull for you…

2 Likes

Oh, yes. It will be useful for sure.

Thanks again! :smiley:

If the deployment fails, it allows to study the compiled compose.yml.

I added the $$(env…) part when I wanted to use a command: declaration that refered to variables inside the container, which envsubst used to replace with null.

1 Like

I missed to add the remove part. It wait until all project ressources are deleted. This prevents race conditions when remove and deploy are executed in a row. Usualy it works fine, but sometimes it gets stuck… well with this, it does never again:

# param 1: docker-compose template file
# param 2: project name of the deployment
define remove-compose
        envsubst "$$(env | sed -e 's/=.*//' -e 's/^/\$$/g')" < $(1) \
                |tee debug.yaml \
                |docker-compose --project-name $(2) -f - down && rm debug.yaml; \
        types="service network config secret"; \
        for type in $$types; do \
                until [ -z "$$(docker $$type ls --filter label=com.docker.compose.project=$(2) -q)" ]; do \
                        sleep 1; \
                done; \
        done;
endef

1 Like