Docker Community Forums

Share and learn in the Docker community.

Alternatives to OSXFS / performant shares under OSX

docker

(Eugenmayer) #1

Since especially this thread was dragged off topic while searching and discussion alternatives, i created this topic to separate this discussions, so people actually interested in getting osxfs fixed do not get bothered by the people actually just looking for a solution, no matter it is osxfs or something else solving this under docker for mac.

An alternative for shares under docker-for-mac is http://docker-sync.io - it does support:

  1. Its performance matches the performance having no shares at all - its as fast as docker is, see this chart

  2. Different strategies to sync and watch for changes: rsync/unison/unison+unox (transparent sync) and will even support NFS in the near future. See a brief description of the possible strategies here

  3. You can chose a strategy to sync with, depending on each project or even for each container you have.
    Since, like in the linked topic above, use cases do differ a lot, you cannot just ask all users to use one single strategy, like most of the other ‘docker-sync’ alternatives do. Docker-sync supports you choosing the right strategy depending on your project size, your project kind and your needs on transperency/speed of sync

  4. Different advanced options to help you justifying it to your use case
    sync_args and more help you introducing flags for your specific use cas

  5. It does support uid/guid mapping to your container, to fix permission issues

  6. It supports being used as a library for scaffolding and implementing your own project based logic, see scripting

  7. It supports per-prroject configuration and also does keep your docker-compose.yml portable by not applying changes to it, all changes go to docker-compose-dev.yml

  8. It supports full-stack bootstrap by starting sync and compose for you by running docker-sync-stack start - fire and forget

  9. It does support notifications on sync so you are aware what happened when

For more informations, check the wiki

As mentions above, there are already enough alternatives you might want to try also - most of them do not support docker for mac at all (except docker-sync-magic, which is part of docker-sync now) or do only implement one strategy to rule them all, which is not working, as a lot of people have different needs. Some of them have still performance issues or do only support one-way syncing and so on


File access in mounted volumes extremely slow, CPU bound
(Fotosys) #2

Works excellent. Thanks for that. Made my day :slight_smile:


(Ross Boucher) #3

You may want to change the link to http (or run your domain through cloudflare), as the current https site serves an invalid ssl cert.


(Eugenmayer) #4

Thank you for the hint @boucher this is actually a gh-page so hosted by github, AFAIK there is no way to use https there. Anyway, changed to https

@fotosys thank you!


(Eugenmayer) #5

http://docker-sync.io just got its 0.1.0 release with a new, full transparent sync strategy, 2-way. It is as transparent as NFS, but runs a lot faster on the container. https://github.com/EugenMayer/docker-sync/releases/tag/0.1.0

With rysnc, unison and unison-onesided docker-sync basically covers all cases you might have, while running at full speed.

If you do not know which to choose, pick unison - its like have a natural share at steroids


(Cameron Eagans) #6

It’s entirely possible to support NFS like Vagrant OOTB. You can already connect to the Docker for Mac VM via the tty socket and the NFS package can be installed through apk. The only thing that’s needed is to set up the export on your host in a way that’s not horrendously insecure and to have some automagic way of configuring the mount inside the DfM VM.

I wish we had legit SSH access to the DfM VM, because in that case, you could add the DfM VM to docker-machine and use docker-machine-nfs.

I understand that NFS has it’s shortcomings, but for some use cases, it’s still a step up from osxfs for the time being.


(Eugenmayer) #7

Well, OSXFS is explicitly not the topic in here, rather all alternatives. I tried NFS, its just way to slow and thats why i am currently no longer consider adding it to docker-sync anymore ( right now ). The new unison strategy in docker-sync, which combines it with unox, does a full transparent sync like NFS, but without any performance degrading at all.

On the bottom line, i cannot see a single advantage in using NFS over unison+unox - if you give me any good arguments, i might consider it in docker-sync in the future.

For now, we have an ultra-fast sync with rsync ( onsided ), and a very robust and fast / CPU low-profile 2 way sync with unison-unox


(Cameron Eagans) #8

NFS might be too slow for you, but it’s fast enough for many people. I tried Docker Sync and the project I’m on just has way too many files for it to work properly. There were also sync conflicts from Unison because our web container ships with a specific release of the project include (so that we can just pull/run the container in production). The workflow right now is that we’re mounting files over the top of that directory, but Unison complains loudly when that’s the case.

In addition, the entire point of containerizing our environment was so that our developers didn’t have to install extra things on their host (only Docker), so Docker Sync is not really an acceptable solution for us at present.

To be very clear, when I say that NFS would work well for us, I’m not suggesting that happen through Docker Sync. I’m suggesting the DfM should have an option to fall back to NFS if we don’t need filesystem notifications. Also, Docker for Windows uses SMB mounts. It’s not ideal, but even that works better for our current project than osxfs (compared Docker for Mac to Docker for Windows and DfW was blazingly fast in comparison). We have no need for inotify type things at present. We just need decent read performance, and with cachefilesd (even with a 10 second timeout), it’s decent enough.


(Eugenmayer) #9

Fair enaugh, if you handle > 2GB of files, unison is slow at the start, thats right.

I fully get the point with zero-dev deps on the OSX host, but be serious, thats not possible right now. Neither docker works well enough under OSX (under linux, docker-sync is useless) but in most cases you end up installing some of the interpreters, be it ruby/php others to have a decent IDE integration, since not anything can be handled using remote-interpreters.

I really like the idea of the “clean OSX host”, but for now, there is a lot to happen that this actually works out ( not only a good solution without something like docker-sync ). That said, i at least expected, that a lot of people will still have a proper non EOL ruby installed ( not 1.9, like in OSX ), and thats basically all you need.

Still, running zero deps, NFS most probably is a good tool and also for huge codebases ( we tracked down huge means rather size and amount, not just amount. 40k files are not issue)


(Cameron Eagans) #10

I disagree. https://github.com/mickaelperrin/docker-magic-sync seems to be an approach that avoids host dependencies. Other than it not working reliably, something like that is exactly what I need. I haven’t had time to test it recently, so maybe it works better now.


(Eugenmayer) #11

mickael is very actively contributing to docker-sync and the “best” out of docker-magic-sync is in docker-sync “unison”. He is not used magic-sync right now also ( too unreliable )

The issue why we did not use “magic-sync” as a strategy is, its to faulty. A lot of OSXFS fsevents are lost, so sync does not happen. So until this issue Inotify events are not / stop working from host to container is not solved, its just not usable. I know that you already had some conversation about this already with him, AFAIR, so you probably already know this.

As soon as it gets fixed, it will be part of docker-sync. When we tackle to go-rewrite even the ruby dependency would be removed, but well, thats about to discussed anyway


(Cameron Eagans) #12

My point is, this is doable without local dependencies or configuration of any sort. The ideal solution for my team is a few extra lines in our docker compose file and that’s it. Magic sync - as a concept - satisfies this requirement, barring the fs events being lost (which is something that needs to be fixed/maybe has been fixed in osxfs).

Even building docker-sync in Go is a nonstarter in my book because it still requires something extra running on the host (though losing the Ruby dependency is nice).


(Cameron Eagans) #13

Just following up here. https://github.com/cweagans/docker-bg-sync is a working alternative to both docker-magic-sync and docker-sync that has no host dependencies. It’s not quite as automatic as docker-magic-sync, but you only have to set it up once and it works. I’m seeing a 10x-11x speedup on my Drupal 8 project (measured via uncached page load time).


(R6by) #14

I tested cweagans/bg-sync over the weekend with Drupal 8 (including loads of additional dependencies managed by Composer), and got the exact same results (> 10x performance improvement measured by ApacheBench). The reason I’m interested in this approach above the others is the same as what’s listed in the why not use x section of the README. Namely, my entire reason for wanting to use Docker is to contain as many dependencies as possible, leaving my host machine as clean as possible. In this case, all the syncing happens from within the bg-sync container. Of course I’d prefer the filesystem latency issues fixed in OSXFS, but until then, this solution is perfect for my use.


(Eugenmayer) #15

has no host dependencies.

Sure it has, unison and unox. Therefor easy-install of python and xcode + gcc for compiling unison or homebrew.
That is for sure not dependency free, that is just cherry picking. Anyway, as long as it works for you, its a go.


(Cameron Eagans) #16

No, Unison runs in a container. There are *no* host dependencies other than docker.


(Eugenmayer) #17

@cweagans oh i see, you are still using OSXFS to sync and then just syncing further to the app-container. Are you aware that this will have issues due to OSXFS not passing all file events, leading to syncs not triggered for unison?

The solution is elegant, but its not reliable, thats the reason why we yet did not implement it in docker-sync.


(David Sheets) #18

We believe osxfs delivers all mutating file system events (those that unison would care about) to the Linux VFS. You can read about the events that osxfs supports and how at https://docs.docker.com/docker-for-mac/osxfs/#/file-system-events. There are no outstanding partial event delivery bugs that we are aware of. If you know of existing issues with osxfs's inotify support, please report those issues to https://github.com/docker/for-mac and we will work to fix them.


(Eugenmayer) #19

@dsheets sorry, i missed that Inotify events are not / stop working from host to container seems to be fixed for now. Well, i am not aware of anything else yet, we most probably give it a try then. Thank you for the update!

@dsheets update: seems like huge performance issues have been occurring: https://github.com/mickaelperrin/docker-magic-sync/issues/1#issuecomment-237881351


(David Sheets) #20

@eugenmayer I can’t do anything with that report of 100% CPU use until I get a lot more details about the use case/reproduction that causes it to occur. There are known performance issues with osxfs and, recently, with the virtual block device. It would be best to supply a reproduction of the excessive CPU use or slow performance in order to help us understand if this is the same as previously reported issues or a new problem.