Data directory "/var/lib/postgresql/data" has wrong ownership... on Linux?

jralfonsog · July 28, 2020, 8:01pm

Hello there.

I’m running a docker-compose stack with the services below:

A Jupyter notebook service
A mlflow tracking server
A postgresql database to stock mlflow info.

When I try to run docker-compose build and up, I get the classical “data directory “/var/lib/postgresql/data” has wrong ownership” that happened on Windows, but I’m running it on Linux and I can’t find the cause. Please, find docker-compose.yml and respective Dockerfile’s on Dropbox - files.tar.gz - Simplify your life (I can’t upload files), and here you can see the full log:

postgres_1 | The files belonging to this database system will be owned by user “postgres”.
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale “en_US.utf8”.
postgres_1 | The default database encoding has accordingly been set to “UTF8”.
postgres_1 | The default text search configuration will be set to “english”.
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data … ok
postgres_1 | creating subdirectories … ok
postgres_1 | selecting dynamic shared memory implementation … posix
postgres_1 | selecting default max_connections … 20
postgres_1 | selecting default shared_buffers … 400kB
postgres_1 | selecting default time zone … Etc/UTC
postgres_1 | creating configuration files … ok
postgres_1 | running bootstrap script … 2020-07-28 19:47:30.936 UTC [73] FATAL: data directory “/var/lib/postgresql/data” has wrong ownership
postgres_1 | 2020-07-28 19:47:30.936 UTC [73] HINT: The server must be started by the user that owns the data directory.
postgres_1 | child process exited with exit code 1
postgres_1 | initdb: removing contents of data directory “/var/lib/postgresql/data”

Any help you could provide would be great. Thanks for your time and dedication.

tekki · July 29, 2020, 6:58am

You could use a named volume for postgres instead. Or is there a specific reason why you want to use the local file system for the data?

jralfonsog · July 29, 2020, 9:40pm

Hi, @tekki, and thanks for your response.

I would rather have it inside the project folder, I’m not sure if I could specify where to store the named volume.

By the way, I’ve tried it with a named volume this time and it’s not working either. But this time I just get

chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted

.

The docker-compose.yml file, in case I’m doing it wrong (if needed, I’ll change env variables into value, but I think the problem is in those last fields which I’ve already changed ):

version: '3.7'
services:
  jupyter:
    user: root
    build:
      context: .
      dockerfile: ./docker/jupyter/Dockerfile
      target: ${JUPYTER_TARGET}
      args: 
        - MLFLOW_ARTIFACT_STORE=/${MLFLOW_ARTIFACT_STORE}
        - MLFLOW_VERSION=${MLFLOW_VERSION}
        - JUPYTER_BASE_IMAGE=${JUPYTER_BASE_IMAGE}
        - JUPYTER_BASE_VERSION=${JUPYTER_BASE_VERSION}
        - JUPYTER_USERNAME=${JUPYTER_USERNAME}
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${JUPYTER_TARGET}:${VERSION}
    ports:
      - "${JUPYTER_PORT}:${JUPYTER_PORT}"
    depends_on: 
      - mlflow
    environment: 
      MLFLOW_TRACKING_URI: ${MLFLOW_TRACKING_URI}
      JUPYTER_ENABLE_LAB: ${JUPYTER_ENABLE_LAB}
      NB_USER: ${JUPYTER_USERNAME}
      NB_UID: ${JUPYTER_UID}
      CHOWN_HOME: "yes"
      CHOWN_HOME_OPTS: '-R'
      CHOWN_EXTRA: ${JUPYTER_CHOWN_EXTRA}
      CHOWN_EXTRA_OPTS: '-R'
    volumes:
      - ./:/home/${JUPYTER_USERNAME}/work
      - ./${MLFLOW_ARTIFACT_STORE}:/${MLFLOW_ARTIFACT_STORE}
  mlflow:
    build:
      context: ./docker/mlflow
      args:
        - MLFLOW_VERSION=${MLFLOW_VERSION}
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${MLFLOW_IMAGE_NAME}:${VERSION}
    expose: 
      - "${MLFLOW_TRACKING_SERVER_PORT}"
    ports:
      - "${MLFLOW_TRACKING_SERVER_PORT}:${MLFLOW_TRACKING_SERVER_PORT}"
    depends_on: 
      - postgres
    environment: 
      MLFLOW_TRACKING_SERVER_HOST: ${MLFLOW_TRACKING_SERVER_HOST}
      MLFLOW_TRACKING_SERVER_PORT: ${MLFLOW_TRACKING_SERVER_PORT}
      MLFLOW_ARTIFACT_STORE: ${MLFLOW_ARTIFACT_STORE}
      MLFLOW_BACKEND_STORE: ${MLFLOW_BACKEND_STORE}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DATABASE: ${POSTGRES_DATABASE}
      POSTGRES_PORT: ${POSTGRES_PORT}
      WAIT_FOR_IT_TIMEOUT: ${WAIT_FOR_IT_TIMEOUT}
    volumes:
      - ./${MLFLOW_ARTIFACT_STORE}:/${MLFLOW_ARTIFACT_STORE}
  postgres:
    user: "${POSTGRES_UID}:${POSTGRES_GID}"
    build:
      context: ./docker/postgres
    image: ${IMAGE_OWNER}/${REPO_SLUG}/${POSTGRES_IMAGE_NAME}:${VERSION}
    restart: always
    environment: 
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_USER: ${POSTGRES_USER}
    ports:
      - "${POSTGRES_PORT}:${POSTGRES_PORT}"
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

By the way, I’m using Docker v19.03.12, build 48a66213fe on Linux Mint 19.03

meyay · July 29, 2020, 10:03pm

… and this is the reason:

jralfonsog · July 29, 2020, 10:10pm

Hey, @meyay, thanks for the quick answer.

I’m running it through a Makefile, which contains:

POSTGRES_UID := $(shell id -u)
POSTGRES_GID := $(shell id -g)

Isn’t it correct? What should I do?

meyay · July 29, 2020, 10:28pm

Is it mentioned in the description of the dockerhub page that the image actualy support/requires to set the user?

jralfonsog · July 29, 2020, 10:32pm

I would say so…

github.com

docker-library/docs/blob/master/postgres/README.md#arbitrary---user-notes

<!--

********************************************************************************

WARNING:

    DO NOT EDIT "postgres/README.md"

    IT IS AUTO-GENERATED

    (from the other files in "postgres/" combined with a set of templates)

********************************************************************************

-->

# Quick reference

-	**Maintained by**:  
	[the PostgreSQL Docker Community](https://github.com/docker-library/postgres)

This file has been truncated. show original

meyay · July 29, 2020, 10:40pm

So according docs it should work for the debian based images. That’s a safe starting point for an analysis.

Since your docker-compose.yml does obfuscate all details about the image:tag, it is impossible to see which tag you actualy used. I guess this is the price you pay if you use make and envsubst to have dynamic templating The both statements from above look like valid Makefile syntax.

You might want to take a look at the entrypoint script for that particular version: https://github.com/docker-library/postgres, you can start your analysis in the _main() function.

jralfonsog · July 29, 2020, 10:52pm

And… I should change the title because I know what was happening #facepalm

I didn’t realize was using for these tests an old folder I had mapped from a NTFS HD (I run 1 HD for Linux, another for Windows and a shared NTFS one where this folder is mapped…).

Seems like the problem, even though I’m running this on Linux, comes from that last NTFS disk. I’ve tried with another location for the named volume on the Ext4 and it runs perfectly.

So thanks a lot for your dedication, and sorry for having you busy with this one

meyay · July 29, 2020, 10:57pm

don’t worry.

Btw. If you ever have the situation that you end up with variables in your compose template, you can trick envsubst in only using variables that actualy exist in env:

# param 1: docker-compose template file
# param 2: project name of the deployment
define deploy-compose
        envsubst "$$(env | sed -e 's/=.*//' -e 's/^/\$$/g')" < $(1) \
                |tee debug.yaml \
                |docker-compose --project-name $(2) -f - up -d  && rm debug.yaml;
endef

Maybe this might be usefull for you…

jralfonsog · July 29, 2020, 10:59pm

Oh, yes. It will be useful for sure.

Thanks again!

meyay · July 29, 2020, 11:01pm

If the deployment fails, it allows to study the compiled compose.yml.

I added the $$(env…) part when I wanted to use a command: declaration that refered to variables inside the container, which envsubst used to replace with null.

meyay · July 29, 2020, 11:12pm

I missed to add the remove part. It wait until all project ressources are deleted. This prevents race conditions when remove and deploy are executed in a row. Usualy it works fine, but sometimes it gets stuck… well with this, it does never again:

# param 1: docker-compose template file
# param 2: project name of the deployment
define remove-compose
        envsubst "$$(env | sed -e 's/=.*//' -e 's/^/\$$/g')" < $(1) \
                |tee debug.yaml \
                |docker-compose --project-name $(2) -f - down && rm debug.yaml; \
        types="service network config secret"; \
        for type in $$types; do \
                until [ -z "$$(docker $$type ls --filter label=com.docker.compose.project=$(2) -q)" ]; do \
                        sleep 1; \
                done; \
        done;
endef

Topic		Replies	Views
Data directory "/var/lib/postgresql/data/pgdata" has wrong ownership Docker Desktop docker , beta , windows	43	217797	August 15, 2023
Data directory "/var/lib/postgresql/data" has invalid permissions Docker Hub ci_cd	2	18569	November 17, 2021
Quickstart: Compose and Rails - postgres permissions error Docker Desktop	0	1783	April 3, 2019
Persistent data on Windows 10 from Linux container Docker Desktop	0	2307	August 28, 2019
Trying to get Postgres to work on persistent windows mount - two issues Docker Desktop windows	7	45705	December 31, 2021

Data directory "/var/lib/postgresql/data" has wrong ownership... on Linux?

Related topics