Share and learn in the Docker community.
Apache 2.4.7 on Ubuntu 14.0.4 inside a Docker container on OSX randomly gives the following 500 error while requesting random static files during a page load:
Error reading /var/www/.htaccess on line 1: Bad file descriptor
This only started happening after updating to Docker 1.11.1-beta13
I can confirm this. When requesting “many” static files during a page load (in my case, just 2 files), my PHP-FPM process gets killed randomly:
This is always reproducible when doing a full page-refresh. When the browser has the assets cached, this problem doesn’t occur because the files don’t need to be downloaded again.
Same, seeing this error a lot in my Rails app after upgrading to beta13.
Errno::EBADF: Bad file descriptor @ io_fread
We’re seeing bad file descriptors as well with a python uwsgi app. Seems like a permissions issue or something. I was getting file descriptor errors once I updated. Then I ran chmod 777 -R * from within our container on the entire repo and then things started working again. Only two of us are using Docker for Mac and we’re both having this issue. One person is using docker toolbox and has no issues and two other people are using ubuntu without an issue. Let me know if I can provide additional information.
I think I was hitting the same issue. For me I was running an ansible playbook from within the container, and about halfway through the role I would start seeing “bad file descriptor” errors then ansible would just fail to find template files. Finally docker would either hang or crash and some supervising daemon would restart the whole VM (at least I think thats what its doing).
I tried capturing some log output but all I see is
May 26 13:45:34 LP-AKINS-OSX Docker[4841] <Notice>: Creating resource Entry(b01cfc4b4fac14650dd690dd67b21a30549a942bd3c6720d39e60050ba211763) May 26 13:45:34 LP-AKINS-OSX Docker[4841] <Notice>: Write offset=0 data=[b01cfc4b4fac14650dd690dd67b21a30549a942bd3c6720d39e60050ba211763:/[bunch of paths]] to file May 26 13:45:34 LP-AKINS-OSX Docker[4841] <Notice>: Volume.start b01cfc4b4fac14650dd690dd67b21a30549a942bd3c6720d39e60050ba211763 (paths = [ bunch of paths ]) May 26 13:45:34 LP-AKINS-OSX Docker[4841] <Notice>: Created instance b01cfc4b4fac14650dd690dd67b21a30549a942bd3c6720d39e60050ba211763 May 26 13:47:32 LP-AKINS-OSX Docker[4841] <Error>: Unknown exception caught: End_of_file May 26 13:47:32 LP-AKINS-OSX Docker[4841] <Error>: Fatal unexpected exception: End_of_file May 26 13:47:32 LP-AKINS-OSX Docker[1362] <Notice>: Reap com.docker.osxfs (pid 4841): exit status 1 May 26 13:47:33 LP-AKINS-OSX Docker[1362] <Notice>: Stop 1 children with order 1: com.docker.driver.amd64-linux (pid 4844) May 26 13:47:33 LP-AKINS-OSX Docker[1362] <Notice>: Signal terminated to com.docker.driver.amd64-linux (pid 4844) May 26 13:47:33 LP-AKINS-OSX Docker[4844] <Notice>: sending SIGTERM to com.docker.hyperkit pid 4850 May 26 13:47:39 LP-AKINS-OSX Docker[4844] <Notice>: virtio-net-ipc: initialising uuid=b4669773-8b10-47b6-be1e-51bc6020e952,path=/Users/akinsley/Library/Containers/com.docker.docker/Data/s50,macfile=/Users/akinsley/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/mac.0
Ultimately I went back to beta 12 as I hit this issue every time I tried to run a playbook.
I just increased my memory allotment for Docker in the advanced settings menu and suddenly everything is working a lot better. Maybe we’re all running out of space within docker? It only defaults to 2GB. And running out of space would explain a bad file descriptor error.
I haven’t thoroughly tested this, but wanted to put it out there before I leave for the day. Thoughts?
Same issue. Docker for Mac 1.11.1-beta-13. LAMP.
From the apache logs
[Thu May 26 23:05:34.558723 2016] [core:alert] [pid 48] [client 172.19.0.1:37452] /var/www/html/.htaccess: Error reading /var/www/html/.htaccess at line 1: Bad file descriptor, referer: http://localhost:8080/sites/default/files/css/css_9lpLl08tZQNbq8WdIqYfCpdvv1EytCL6xZ3e3X2P2Os.css [Thu May 26 23:05:34.585398 2016] [core:alert] [pid 51] [client 172.19.0.1:37440] /var/www/html/.htaccess: Error reading /var/www/html/.htaccess at line 103: Bad file descriptor, referer: http://localhost:8080/environmental-field-station/explore
Increased memory to 12G from 8G - no dice.
2GB is a ton of memory! What kind of mac are you running that you can allocate TWELVE GIGABYTES to Docker?
You’re right, 2GB is a ton of memory for Docker. We’ve got about a dozen containers we’re spinning up and pretty large code base. I increased it to 8GB and noticed better performance. Today I’m putting it back down to 4GB and we’ll see if there’s any noticeable difference.
I’m getting the same errors. On a long make, not consistent, it seems to happen after the compilation goes for a while. I’ve got 4GB allocated to docker, so will try more.
Also check permissions. Is this related to the UID/GID ticket? If it’s not readable you’d get bad file descriptor.
Sorry what’s the UID/GID ticket, is there a github reference. It is strange because it is doing lib tool or gcc against a long list of files and then one fails randomly.
Thanks for the points @stupschwartz, that does not look like the problem. All the GID and UID are correct (this is for a developer image so we match the outside and inside GID/UID in the Dockerfile).
It sure looks like the file system is not keeping up. the offending command from the make runs fine if I manually type it in. It looks like it is timing related. Running El Capitan 10.11.5 and Docker 1.11.1-beta13, but I don’t know if it worked before then. Is there an easy way to find old betas to see if this is a regression or has always been there.
Finally the image does run fine in native Ubuntu or in VMware Fusion. It does not run properly when docker-machine is used (we do not use Docker Toolbox, so I haven’t tested it). That’s a problem with the script itself that does the setup. Pretty involved for us as we are creating multiple architectures and debug and release variants but that’s unrelated to this problem I think.
Timing bug would explain a lot of the other issues I’m seeing. Including this other ticket I put in ERROR: Bad Response from Docker Engine
Is this where we notify folks. Is there a github issue we should open like with the core docker development?
Hi Richard,
We are working on a fix and will push out an update soon.
Thanks for participating in the Docker for Mac Beta!
Best regards,
David Sheets
Seems like something might’ve broken from the beta13.1 hotfix, I’m now getting ENOENT on files which are there in volumes.
Hi Peter,
Could you please be more specific about the circumstances in which you are seeing ENOENT for files which exist in OS X and are bind mounted into a container? A Dockerfile, shell script, makefile, written instructions, or version control repository demonstrating this issue would be extremely helpful.
Thanks,
David
Hi David
I’m able to reproduce in newly created directories, but not always when rerunning in an emptied directory:
$ mkdir foo; cd foo; docker run -it --rm=true -v "pwd":/root node bash -c "cd \$HOME; npm i phantomjs-prebuilt"
Example error:
Extracting tar contents (via spawned process) Removing /root/node_modules/phantomjs-prebuilt/lib/phantom Copying extracted folder /root/node_modules/phantomjs-prebuilt/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1465222235140/phantomjs-2.1.1-linux-x86_64 -> /root/node_modules/phantomjs-prebuilt/lib/phantom Phantom installation failed { Error: EBADF: bad file descriptor, chmod '/root/node_modules/phantomjs-prebuilt/lib/phantom/third-party.txt' at Error (native) errno: -9, code: 'EBADF', syscall: 'chmod', path: '/root/node_modules/phantomjs-prebuilt/lib/phantom/third-party.txt' } Error: EBADF: bad file descriptor, chmod '/root/node_modules/phantomjs-prebuilt/lib/phantom/third-party.txt' at Error (native) npm info lifecycle phantomjs-prebuilt@2.1.7~install: Failed to exec install script
