Initdb: could not create directory "/var/lib/postgresql/data/pg_xlog": Permission denied

General
1

Hello everyone

I’m not sure if this is a docker issue, or not, but I wanted to post up here anyway, see if I can get some assistance.

I’m on MAC OS X 10.11.3, Docker version 1.10.3, build 20f81dd, docker-machine version 0.7.0-rc1, build 0fb68ca, virtualbox 5.0.16 r105871. This was also occurring on docker-machine version 0.6, I upgraded trying to fix it.

Basically, if I try to mount local volumes to persist data, like this

volumes:
- /Users/mjdavies/pgdata:/var/lib/postgresql/data

I get this

pcstudents_db_1 | fixing permissions on existing directory /var/lib/postgresql/data … ok
pcstudents_db_1 | initdb: could not create directory “/var/lib/postgresql/data/pg_xlog”: Permission denied

There seems to be a lot of issues on github about this, there’s little point putting them in here as there’s so many.

I’ve tried deleting the virtual machine, deleting ~/.docker, everything I can see apart from using this

I don’t really want to have to use something else, as I’m about to have to show 10 developers who have no experience of using docker how to use docker on their macs, and need to keep this all as simple as possible.

Could anyone shed any light on who needs to fix this? Is it mac, is it docker, is it virtualbox, is it all of them?

If anyone else has come up with a solution then please let me know, bit frustrating to bump into this to be honest.

2

It’s VirtualBox shared folders permissions issue. When mounting the shared folder into a container normally the container can access everything in any manner it wants because usually containers are run as root. With PostgreSQL and some other official images they drop down to a less privileged user for most of their lifecycle so their user is not privileged enough to write to that volume.

You need to either unmount and re-mount the vboxfs with new uid / guid mappings (brittle), or figure out a different way to achieve your goal.

It’s worth asking why you want to have a bind mount into the Postgres image in the first place. If you can skip it (or mount it to a native Linux directory instead, e.g. /var/lib/postgresql in the boot2docker VM), you’ll probably be better off.

Alternatively, the Docker for Mac Beta is meant to resolve some of these problems (and they would like to hear from users who have them), so you might want to try and snag an invite.

1 Like
3

Hi Nathan

Thanks for the reply

I’ve registered for the new beta, looks great. Fingers crossed.

I’m trying to achieve an easy way for us all to manage some persistent data sources.

I’ve had a look at mounting a volume in a container, or mounting the volume in the same container, but neither of those options will get around an issue I think we’re going to run into as a development team. I could be wrong though.

The dev team’s skills attributes will vary wildly, some of the designers as an example, whilst being excellent at design, will not be very good at understanding the concepts of containers and images, and will make mistakes, like we all do.

Whilst working though those mistakes with one or two of the devs, and trying to remotely help them, the easiest answer a lot of the times has been to remove all the images and containers, sometimes the ‘default’ host itself.(rare that one, but has happened).

I’ve got a feeling that as I get a bit more proficient at using docker and all the tools, that situation will not happen as often, but it is going to happen. What do you think, am I fussing over nothing, or just being too kind to them and they can always upload a dump of the data from the production database to the new database, if we ever manage to delete everything?

Is there a better docker method to achieve what I want that you know of? Maybe I’m misunderstanding the basic concepts of how Volumes work in docker.

I’m going to have a sqiuz at the docker-machine.nfs stuff, and see how much more complication that adds to the docs, it might not be so bad.

Thanks again

4

No doubt that this happens, but how is having the ability to poke around in /var/lib/postgresql going to help them when it does?

In other words, how does the database get instantiated in the first place? I’d expect an application to have some sort of DB schema patch + initial data seed logic, it should be applied in the container in this case just the same as it would anywhere else.

Just like you’d have to call rake db:migrate; rake db:seed if working on a Rails app with a database from scratch, you’d have to do the equivalent for your app here.

I’d tell you not to develop using production data in development anyway (no reason to do so if you can mock the data instead), but as I said, they should be able to blow away and re-build their data layer using something like make clean; make dbseed (or equivalent) just as easily as you can docker-compose rm -f; docker-compose up.

If you aren’t 100% sure that you need NFS (e.g. for dealbreaker performance reasons), I’d recommend against using it. It will add more complexity, points of failure, and has its own layer of permissions headaches.

5

Hi Nathan

Thanks for getting back to me.

I’m going to have a go at creating a container for the databases in all the apps, so one database container, as a service, per application.

Rebuilding and populating the database layer, like you say, should be easily achieved with seeding. I can pass that element back to the developers to achieve, if they haven’t already.

I’m sure we’ll run into some issues, but I’m sure we’ll be able to cross those bridges as they come along.

Thanks for your help with this.

6

Hi @nathanleclaire ,

We are experiencing the same problems on Mac dev. machines, trying to run postgres:9.5.2 and mongo:3.2.4 on both VMWare Fusion and VirtualBox b2d vm.

Anyway, we are building a complex docker infrastructure on Google Cloud for Car Sharing (http://www.cupcarsharing.com)
The development step, is an essential point to our work pipeline.

I would be really grateful if you could add me to the beta group for both Mac and Windows, or give us a solution for this vexing issue.

Thanks!

7

@mchiang FYI ^^ @maxdl is interested in the Beta

8

Hi @maxdl, please signup at https://beta.docker.com and privately send me your Docker ID to mchiang@docker.com. I’m sure we can work something out provided you are willing to give us feedback.

Thanks for being a Docker user!

Michael

9

I hope it’s ok to jump into the discussion.
I’m trying to do the same volume mount as the postgres official image documentation suggests for persistent data.
So in my version 3 docker-compose yml file I’ve added the following:
volumes:
- ./services-config/postgres/postgres-data:/var/lib/postgresql/data/pgdata

And also added an environment variable:
PGDATA=/var/lib/postgresql/data/pgdata

And in the build process I’m getting:
fixing permissions on existing directory /var/lib/postgresql/data/pgdata … ok
postgres | initdb: could not create directory “/var/lib/postgresql/data/pgdata/pg_xlog”: Permission denied
postgres | initdb: removing contents of data directory “/var/lib/postgresql/data/pgdata”

And then the container exits.

I think it’s the same issue that @vanderkerkoff had, any news regarding this thread?

2 Likes