Docker Community Forums

Share and learn in the Docker community.

Bug: Postgres-based image doesn't run on Docker for Mac, runs fine on linux

bugfixes
docker
ci

(Aviv Rosenberg) #1

General description

I have created an Image based on the official postgres:9.5.3 image.
This image runs fine on docker in linux (tested with ver 1.11.2), but fails to run on Docker for Mac beta (1.12.0-rc4-beta20, diagnostic id = 100BE54F-2248-4431-9382-6BA42F9FA120).

This feels like a serious bug since the premise of docker is that an image, once created, should run exactly the same everywhere…

Information

The purpose of said image is to serve as a pre-seeded DB for tests running in a CI server.
During tests, I want to start a DB container that contains quite a lot of data, allow the test to read and write to it, and then after the test I want all the modifications discarded so they don’t affect the next test that uses this image.

Basically the image uses a non-volume directory as PGDATA, containing data that was copied into the container with docker cp. See “Steps to reproduce” for a full description of how the image was created.

The owner of the custom PGDATA directory is root (because it was copied in).
This is not an issue normally, because the postgres official image comes with an entrypoint script that starts with the following:

mkdir -p "$PGDATA" chmod 700 "$PGDATA" chown -R postgres "$PGDATA"

Expected behavior

The image should run. Like I said, it runs fine on a linux version of docker.
After the entrypoint script is executed, the permissions of PGDATA are correctly set to the postgres user.
Then the database loads without issue and is works fine (I can connect, read write data).

Actual behavior

On Docker for mac, an error is displayed soon after starting the container:

FATAL: could not open lock file "postmaster.pid": Permission denied

Investigating this, I did the following:

  1. Run a container based on the image with bash as entrypoint:
    docker run --rm -it --entrypoint=/bin/bash avivrosenberg/test-db-bug
  2. ls -al $PGDATA, see that everything is owned by root.
  3. Change the permissions of PGDATA like they do in the official entrypoint script:
    chmod 700 "$PGDATA" && chown -R postgres "$PGDATA"
  4. ls -al $PGDATA, see that everything is now owned by postgres.
  5. Try to write to the PGDATA directory as the postgres user:
    gosu postgres touch $PGDATA/foo.bar

This leads me to:

touch: cannot touch ‘/var/lib/postgresql/test-data/foo.bar’: Permission denied

So basically, the touch command I used failed for the same reason as the postgres process trying to create postmaster.pid in the PGDATA directory after changing permissions.

Steps to reproduce the behavior

What I did is basically:

  1. Start a container based on postgres:9.5.3, with custom PGDATA.
  2. Write some data into the database using some program that connected to it. This is optional, not necessary for reproducing the bug.
  3. Stop the postgres process in the container so that everything is persisted to disk.
  4. Create (but don’t start) a second postgres container, set custom PGDATA.
  5. Use docker cp to stream the PGDATA dir from the first container to the second, like so:
    docker cp $CONTAINER1:<path> - | docker cp - $CONTAINER2:<path>
  6. docker commit the second container a new image.

The resulting image is the one that doesn’t run in Docker for mac.

Note:
The reason I’m creating the second container and using docker cp (instead of just committing the first one) is that i’m actually creating multiple DB snapshots, each based on the previous one. I don’t commit directly because then another layer is added and image size becomes a big issue.

To try it:

  • The problematic image can be pulled from dockerhub: docker pull avivrosenberg/test-db-bug.
  • This gist is a script that generates a minimal image which reproduces the bug.

(Ryan Graham) #2

I wonder if this is related to the postgres image declaring a volume at /var/lib/postgresql/data and Docker for Mac mapping that to a shared folder on the host.


(Aviv Rosenberg) #3

I don’t think it’s related since, like I said, i’m using a NON-volume directory as PGDATA in this image. So the permissions problem happens within the filesystem of the image itself, not a volume…


(Ryan Graham) #4

Oops! That’ll teach me for reading forums on my phone.


(Aviv Rosenberg) #5

This issue is still relevant for the latest docker for mac, 1.12.0-beta21 (build: 11019).

@DockerSupport, any chance of looking into this?
Thanks.