Docker Community Forums

Share and learn in the Docker community.

Data directory "/var/lib/postgresql/data/pgdata" has wrong ownership

That’s probably because in this case the volume is created in a path owned by Docker (I think).

This is not a problem if used with Docker Toolbox by the way.

I am also experiencing this issue when attempting to run Postgres within a Docker container. I did a bit of digging and it appears that the Docker for Windows client (1.12.0-rc4-beta19) is mounting the volumes using CIFS and is passing a slew of options:

//10.0.75.1/D
/tmp/wtf
cifs
rw,
relatime,
vers=1.0,
cache=strict,
username=user,
domain=WORKSTATION,
uid=0,
noforceuid,
gid=0,
noforcegid,
addr=10.0.75.1,
file_mode=0755,
dir_mode=0755,
iocharset=utf8,
nounix,
serverino,
mapposix,
noperm,
rsize=61440,
wsize=65536,
actimeo=1
0 0

Of particular interest are uid, gid, noforceuid, noforcegid, and nounix. I haven’t poked around at this yet but it seems as if we are getting stuck with uid=0,gid=0 which corresponds to root:root (as @greut pointed out).

In my experience mapping uid/gid between Windows and Linux can be pretty painful due to differences between the POSIX and NTFS ACL schemes. One potential solution could be more granular management of shares and mount options. I would be very interested to hear others’ opinions on this issue and any potential solutions.

1 Like

what you’re doing is a local volume (which will belong to root), use a docker data volume. What you’re doing is a host directory as a volume. https://docs.docker.com/engine/tutorials/dockervolumes/

1 Like

But how do I create a data volume mapped to a local volume on Docker Compose? I need to persist the database server info so I don’t lose the information each time I run the container.

version: '2'
services:
  db:
     image: postgres:9.4
     volumes:
        - postgres:/var/lib/postgresql/data
volumes:
  postgres:   

But in this case it will create the local volume on Docker folder, right? If so, can’t I map to a different place?

Try another plugin then: https://docs.docker.com/engine/extend/plugins/#/volume-plugins

1 Like

Hello

Same problem with new version 1.12.0-beta21 (build: 5971) 5b59d37

When I don’t map volume like this:

  • /var/lib/postgresql/data
    it works

But when I map a volume like this:

  • .data/postgresql:/var/lib/postgresql/data
    it doesn’t work

It is frustrating because each time I update docker for windows I loose data. I need to backup before update and restore my database after update.

1 Like

Issue is valid even in Stable version. :frowning: Anybody any idea how to workaround this?

Yep same issue here. I’ve tried doing it in my User directory on my C: drive instead but ends up with:

db_1     | syncing data to disk ... initdb: could not fsync file "/var/lib/postgresql/data/base/1": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/base/12374": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/base/12379": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/base": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/global": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_clog": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_commit_ts": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_dynshmem": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_logical/mappings": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_logical/snapshots": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_logical": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_multixact/members": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_multixact/offsets": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_multixact": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_notify": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_replslot": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_serial": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_snapshots": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_stat": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_stat_tmp": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_subtrans": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_tblspc": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_twophase": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_xlog/archive_status": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_xlog": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data": Invalid argument
db_1     | initdb: could not fsync file "/var/lib/postgresql/data/pg_tblspc": Invalid argument

Then

db_1     | waiting for server to start....FATAL:  data directory "/var/lib/postgresql/data" has wrong ownership
db_1     | HINT:  The server must be started by the user that owns the data directory.

It does seem to create files and folders but just won’t start based on permissions

Are there any workarounds?

Are there any workarounds?

Workaround: add the environment variable PGDATA: /tmp to your docker-compose file.

Here’s mine:

version: '2'

services:
  postgres:
    image: postgres:latest
    #tty: true
    ports:
      - "5432:5432"
    volumes:
      - f:/data/docker/postgresql/data:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: docker
      POSTGRES_DB: db
      PGDATA: /tmp
2 Likes

@walkandride Here, we are mounting /var/lib/postgresql/data but using /tmp as PGDATA directory. This didn’t share the data directory with host system.

volumes:
- f:/data/docker/postgresql/data:/var/lib/postgresql/data
environment:
PGDATA: /tmp

Is there some workaround to deal with this? I am using postgres:9.4 tag, is this issue common for windows across all versions?

Hi Vishal,
In your example, “/var/lib/postgresql/data” should be the directory on your host [Windows] system. I don’t think you have this directory in Windows.

In my example, I have a drive F: and map a host directory on this drive “f:/data/docker/postgresql/data” to the postgres data directory /var/lib/postgresql/data. Setting the PGDATA environment variable resolves the fsync errors that I received.

Hope this helps.

  • John

Thank you for quick response. In the snippet I used from your quote, PGDATA points to “/tmp” which would mean PG will use /tmp as data directory inside container. I assume then PG doesn’t use “/var/lib/postgresql/data” directory as data directory.

In that case, I am not very sure if mapping “/var/lib/postgresql/data” container directory to x directory on host system would do any help. Doing this surely resolves all errors, but the “f:/data/docker/postgresql/data” directory on host system doesn’t really reflect the up-to-date data of PG, because it is now residing in /tmp.

As I am still learning docker, am I missing some point here?

Thank you in advance.

When you’re using Docker for Windows to volume-mount a Windows drive into a Linux container, that volume is done using a CIFS/Samba network share from the Windows host. For lots of reasons, it’s highly unlikely that Linux Postgres will work correctly when trying to write data to a filesystem backed by NTFS shared with Samba.

Instead, I recommend using a persistent (but local to the Linux VM) named volume as detailed here: Trying to get Postgres to work on persistent windows mount - two issues

2 Likes

Hi Vishal,
I’m still learning Docker too. I verified that you are correct. After I did a “docker-compose down”, I no longer had my data.

The response by Michael above provided the correct solution. I updated my docker compose file to be:

version: '2'

# docker volume create --name data -d local

services:
  postgres:
    restart: always
    container_name: postgres_db
    image: postgres:latest
    #tty: true
    ports:
      - "5432:5432"
    volumes:
       - data:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: docker
      POSTGRES_DB: db
      #PGDATA: /tmp

volumes:
  data:
    external: true

After, creating the “data” volume, I did a docker-compose. Everything initialized fine (no errors). I then created a database and loaded it with data. I then did a docker-compose down followed by a docker-compose up and the data was still there.

The thing that I am unsure of is where on the disk is this data stored. docker volume inspect does not provide any meaningful information.

1 Like

I had exact issue with postgres:9.4, my manifest file looks quite similar:

version: '2'
services:
  postgres:
    image: "postgres:9.4"
    ports:
     - "65432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
volumes:
  pgdata:
    driver: local

when I run docker-compose up I got following errors:

postgres_1  | FATAL:  "/var/lib/postgresql/data" is not a valid data directory
postgres_1  | DETAIL:  File "/var/lib/postgresql/data/PG_VERSION" does not contain valid data.
postgres_1  | HINT:  You might need to initdb.

I confirm that this issue affect version 9.5 and 9.6. The only version that finally get this issue rectified is version 10.

So if you could not afford to use version 10, here is my fix:

version: '2'
services:
  postgres:
    build:
      context: ./
      dockerfile: Dockerfile.postgres
    ports:
     - "65432:5432"
    environment:
      - PGDATA=/var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
volumes:
  pgdata:
    driver: local

and the content of Dockerfile.postgres is:

FROM postgres:9.4

RUN mkdir -p "$PGDATA" && chmod 700 "$PGDATA"

The gist is to explicitly specifiy PGDATA and ask docker-compose to use my custom Dockerfile.postgres file to build the image. The RUN rule is very self-explanatory, I create a new folder and change mod to 700. And viola, it works :slight_smile:

2 Likes