File system watch does not work with mounted volumes

Expected behavior

File system notification works with mounted directories.

Actual behavior

No notification is perceived.

Information

  • docker beta, version 1.11.1-beta11 (build 2789) b0bc231

Steps to reproduce the behavior

  1. mount a volume, like so: docker run -ti -v C:\proj:/app <your_image> /bin/bash
  2. Install inotify-tools: apt-get update && apt-get install inotify-tools
  3. Start the notification: while inotifywait -e close_write Startup.cs; do echo updated; done
  4. Start bash from another shell: docker exec -ti <container_name> /bin/bash
  5. Touch a file in the second shell, the notification shows up on the first shell: touch Startup.cs
  6. Change the file in Windows, no notification is sent to the first shell
5 Likes

IMO the windows mount is done with CIFS, which may not support the mechanisms needed to have low level IO functions.
see: https://www.google.at/search?q=+inotify-tools+cif&ie=utf-8&oe=utf-8&gfe_rd=cr&ei=z9E4V4CsCsuk8we72orQCg#q=inotify-tools+cifs+problem&tbas=0

pmario, do you have a link that documents this is the case? As of samba 3.0.25 inotify events should be supported: https://wiki.samba.org/index.php/Samba_3.0_Features_added/changed

I don’t think the relevant code is in the Linux kernel any longer:

Hi Michael,
In May I was just guessing, how the stuff works, from what I could see, since there was (still is) no accessible source code. https://youtu.be/7da-B3rY9V4?t=2094 here’s the proof. Windows is different :slight_smile:

Have there been any developments on this front? Seems like a pretty critical feature missing from windows for this long.

Sorry, no progress to report. Docker for Windows relies on SMB/CIFS support in Linux, and since propagating filesystem notifications is not supported our options are limited.

I made a really gross hack which just scans a directory of files recursively, over and over, looking at the modified times of files and if they changed since the last scan it fires an event. The file system events don’t fire but the files and their metadata do change successfully, so this technique can work. It’s pretty poor performance but for a dev-time only script it’s a lot better than dockering down then back up just to pickup file changes.

It’s a simple nodejs/gulp script if anyone wants it let me know and I’ll share it.

What programming language and framework are you using? Most dev web servers will have fallback mode that resort to filesystem polling in cases like this.

I have developed the script that sits on Windows host, watches changes in directories mounted by Docker containers and notifies containers once file change occurs. The script is available as pip package (PyPI)

pip install docker-windows-volume-watcher

Then you can just run the script without any arguments:

docker-volume-watcher

The script will inspect all running containers and start notifying containers about changes in mounted directories. The script will also listen container start/stop events and update the list of watched directories.

You can read detailed description of this script in this blog post.

3 Likes

docker-volume-watcher works great for me! It would be great having it integrated inside Docker for Windows. Mikhail, thank you very much for developing this util.

I would love to try your script, has to be better than manually rebuilding all the time. For some reason @merofeev solution just gives me an error.

I used this in gulp4 to work around the issue.

Usage:

import gulp from 'gulp'
import gls from 'gulp-live-server'
import { build } from './build'
import { scan } from './ghetto-watch'

let _server = gls(['-r', 'babel-register', 'index.js'])

function start (callback) {
  _server.start()
  callback()
}

function stop (callback) {
  _server.stop()
  if (callback) callback()
}

function notify (callback) {
  _server.notify()
  callback()
}

function watch () {
  let src = [
    '*.js',
    'api/**/*',
    'site/**/*'
  ]

  let scanner = scan(src)
  let full = gulp.series(
    scanner.stop.bind(scanner),
    build,
    notify,
    stop,
    start,
    scanner.start.bind(scanner)
  )

  scanner.on('changed', (changed) => {
    console.log('changed:', changed)
    full()
  })
  scanner.start()
}

const run = gulp.series(build, start, watch)
gulp.task('start', run)
gulp.task('default', run)
// ghetto-watch.js
import fs from 'fs'
import path from 'path'
import map from 'async/map'
import mm from 'micromatch'
import { EventEmitter } from 'events'

// This should not exist but docker volumes for windows do not trigger file system events
// So instead I scan all the files looking for modify time changes and manually trigger events

let cache = {}
let token = null

function flatten(arr) {
  return arr.reduce(function (flat, toFlatten) {
    return flat.concat(Array.isArray(toFlatten) ? flatten(toFlatten) : toFlatten);
  }, []);
}

function scanFiles (file, callback) {
  fs.lstat(file, (err, stat) => {
    if (err) return callback(err)
    if (stat.isDirectory()) {
      let dir = file
      fs.readdir(dir, (err, files) => {
        if (err) return callback(err)
        let resolved = files.map(f => path.join(dir, f))
        map(resolved, scanFiles, (err, results) => {
          if (err) return callback(err)
          let changed = flatten(results.filter(f => f))
          callback(null, changed)
        })
      })
    } else if (stat.isFile()) {
      let mtime = stat.mtime.getTime()
      let changed = cache[file] != null && cache[file] !== mtime
      cache[file] = mtime
      callback(null, changed ? file : null)
    } else {
      callback()
    }
  })
}

export class Scanner extends EventEmitter {
  constructor (src) {
    super()
    this.running = false
    this.src = src.map(p => path.resolve(p))
  }

  scan (file, callback) {
    if (!this.running) return callback()
    fs.lstat(file, (err, stat) => {
      if (err) return callback(err)
      if (stat.isDirectory()) {
        let dir = file
        fs.readdir(dir, (err, files) => {
          if (err) return callback(err)
          let resolved = files.map(f => path.join(dir, f))
          map(resolved, scanFiles, (err, results) => {
            if (err) return callback(err)
            let changed = flatten(results.filter(f => f))
            callback(null, changed)
          })
        })
      } else if (stat.isFile()) {
        let mtime = stat.mtime.getTime()
        let changed = cache[file] != null && cache[file] !== mtime
        cache[file] = mtime
        callback(null, changed ? file : null)
      } else {
        callback()
      }
    })
  }

  run () {
    if (this.running) {
      this.scan(process.cwd(), (err, changed) => {
        if (err) console.error(err)
        if (mm(changed, this.src).length) this.emit('changed') // a change matching src glob happened!
        setTimeout(() => this.run(), 0)
      })
    }
  }

  start (callback) {
    if (!this.running) {
      this.running = true
      this.run()
    }
    if (callback) callback()
  }

  stop (callback) {
    this.running = false
    if (callback) callback()
  }
}

export function scan (src) {
  return new Scanner(src)
}

Thank you, will see if I get it to work for me :slight_smile:

it’s work for me, thx!

Hi,

Is there any plan to mock this feature, by implementing something like docker-windows-volume-watcher inside docker for windows ?

That’s a pretty big deal for developers

Edit: Apparently LCOW could solve this issue, any news about that ?

how come this isn’t a show stopper?

1 Like

It seems nobody gives a sh*t

Hello, I definitely give a sh*t. However, this person’s python script has saved my bacon: https://github.com/merofeev/docker-windows-volume-watcher

It basically runs in the background and propagates all mounted volume filesystem changes to your running docker containers.

1 Like

Is anyone able to verify if WSL2 solves this problem?

1 Like