Docker Community Forums

Share and learn in the Docker community.

Alpine repos have limited support for kernel-devs

docker
beta

(Dhananjay Sampath) #1

Context If my docker app were to require a kernel module, I’d have to build my module on the same environment as the base VM that the Docker Mac Beta runs on. Currently the base VM is Alpine linux Kernel version 4.4.6 and I don’t have a choice to pick another distro. I’d like to have that choice.

Expected behavior

I’d like to be able to run specific versions of libs (gcc, libc, openssl…) when I build my kernel modules for my app (hobby app: trying to run my raspberry pi based coffee machine on docker!). With docker-machine (running distro of my choice - b2d (debian, tce), ubuntu, centos), I could do this easily. I expect to do the same w “Docker mac”

Actual behavior

Turns out, Alpine package manager has limited set of packages forcing devs to build these from source. This can be tedious and cause a significant degradation in user experience for folks moving to Docker to build their apps.

Caveat

  1. While this can easily be done using a linux box, I use Mac to do my development
  2. I think Alpine linux is awesome to run your final app (even in prod). As a dev I’d rather not run it as my base environment.
  3. Mac for beta is pretty nifty, certainly consumes less resources, results in longer battery life. I get the trade-off between all of that goodness and the slimmed down version of the distro running underneath. I’d just like to have the choice when I start doing kernel work.

Information

(This is not a bug but a feature request)

Steps to reproduce the behavior

  1. Just try to build any kernel driver (that requires gcc-4.9) in an alpine container on top of docker mac beta

(Justin Cormack) #2

We do not intend to support running arbitrary distributions in docker4mac. We need to have a single environment that we can support. I do not understand your question though, you can build kernel modules in a container, the base distro has no effect. Also you should really not need to add custom kernel modules to control a coffee machine, you should be able to do it fine from user space.


(Dhananjay Sampath) #3

We do not intend to support running arbitrary distributions in docker4mac.

Well, that answers my question. That’s too bad. I see most devs sticking to running it using docker-machine + virtualbox in that case. :frowning: (or move to a linux distro entirely).

I do not understand your question though, you can build kernel modules in a container, the base distro has no effect.

Right. Now try testing them on it (you’d have to insert your kernel module that is now dependent on the base distro).

Also you should really not need to add custom kernel modules to control a coffee machine, you should be able to do it fine from user space.

(Gasp!) It is hard for me to get you to try out my custom setup here.

But I can do one better. I’ll give you a simpler, more concrete example that you can try out yourself. You might have heard of a monitoring tool called “sysdig” (www.sysdig.org). Could I get you to run sysdig on docker4mac? Here is the quickest way to run it:

$> docker pull sysdig/sysdig
$> docker run -i -t --name sysdig --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro sysdig/sysdig

fwiw: I can run it perfectly fine with my base distro being b2d/ubuntu (using docker-machine).

I have similar problems testing my custom wifi-kernel module on docker4mac (my coffee machine runs stripped down version of ubuntu)

Hope this helps clarify my question.


(Justin Cormack) #4

Kernel modules are not dependent on the base distro in any way. You just need to compile them against the correct headers, eg 4.4.6, the upstream version should be fine. You will never be able to test your wifi module as there is no wifi hardware in the VM, but you should be able to build and run sysdig.


(Dhananjay Sampath) #5

Here is what happens when you try running sysdig:

`$ docker run -i -t --name sysdig --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro sysdig/sysdig

  • Setting up /usr/src links from host
    ls: cannot access ‘/host/usr/src’: No such file or directory
  • Unloading sysdig-probe, if present
    rmmod: ERROR: …/libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.4.6-moby/modules.builtin.bin’
    rmmod: ERROR: Module sysdig_probe is not currently loaded
  • Running dkms autoinstall
    Error! echo
    Your kernel headers for kernel 4.4.6-moby cannot be found at
    /lib/modules/4.4.6-moby/build or /lib/modules/4.4.6-moby/source.
  • Trying to load a system sysdig-probe, if present
  • Trying to load a dkms sysdig-probe, if present
  • Trying to find precompiled sysdig-probe for 4.4.6-moby
    Found kernel config at /proc/config.gz
  • Trying to download precompiled module from https://s3.amazonaws.com/download.draios.com/stable/sysdig-probe-binaries/sysdig-probe-0.9.0-x86_64-4.4.6-moby-a4e3e87db30214d0d22c479154efe327.ko
    Download failed, consider compiling your own sysdig-probe and loading it or getting in touch with the sysdig community`

For my wifi - I emulate my wifi device (Edimax EW-7811Un ) so it works just fine on the VM.

Bottomline: I got 'em all to work. There is dependency on the base VM to be able to dev/test kernel drivers. It worked well with the docker-machine (generic) driver option. Alpine forces me to build my dependencies from source since the package manager is limited. I was hoping that there would be an option to pick my base distro. Looks like there isn’t a plan. That’s too bad.


(Dhananjay Sampath) #6

Also, fwiw: tagging kernel as 4.4.6-moby doesn’t seem like a great idea. It might be better to leave it as the generic one shipped by Alpine.


(Etown) #7

Check out https://medium.com/@etown/sysdig-running-on-the-new-native-docker-apps-cbc274f80cc0#.ngfpehlnl


(Justin Cormack) #8

Thanks for building that!

The next beta will have a new kernel (4.4.9) so the image will need to be updated.


(Etown) #9

No problem. thanks for the beta. Loving it!

The image grabs the version and downloads the kernel, so as long as the kernel configuration is still available it should still work after the update.


(Justin Cormack) #10

Ok cool. Yes, we will always keep the config available.


(Foucault DE BONNEVAL) #11

Hi guys,

recently spent some time to find a solution to got sysdig/csysdig running on Docker for mac.
After a long battle and exploration of the great internet I now have a way to have it run.

I did not plug the build into the hub as I would have to generate an image per Moby kernel version.
The idea is for each user to make his own local build.

All the process is wrapped into a Makefile, just
# make build
Wait like 15’, watch your Mac getting high in CPU and
# make
To get into csysdig

Project is hosted at : https://github.com/fdebonneval/sysdig-moby

I’d love some feedbacks around this little project, and please fork and help :slight_smile:

Foucault