Creating Docker binary from moby and install on another machine

Hi, I wanted to customize docker so I checked out the moby/moby github repository and added my code changes to it. I’d now like to build a docker binary from this which I can install as a fully fledged working docker on another setup similar to just as I would install docker usually. But, I’m not sure how to go about doing it. Can anyone share the steps or point me to the right reference if anyone has an idea about this?

You should have docker already installed (if not, go ahead and install it) on your system before you try to compile new binaries. Next step, after making the required code changes, is to run the following command from moby base folder:

make binary
(this step might take long time when you run it the first time depending on your internet connection)

This will go through the make process and new binaries will be saved to the folder ./bundles/binary-daemon

This finishes the “How to compile the binaries”

Next is to make your currently installed docker client use this newly compiled binaries for daemon.

For this rename or back-up the old dockerd file you have in /usr/bin folder (or do whereis dockerd on terminal to find path and rename it), and add the path where the binaries were created to system variable $PATH:

export PATH=$PATH:/path/to/moby_base_folder/bundles/binary-daemon

Next step would be to kill ongoing docker process, and restart docker service ( i use these commands, because sometimes “service docker restart” does not work on my system. If it works directly, that one line of code should be enough):

rm -rf /var/run/docker.pid
ps axf | grep docker | grep -v grep | awk ‘{print "kill -9 " $1}’ | sudo sh
service docker restart

Now the docker client will contact the new docker daemon you created.

1 Like

Thank you for the detailed explanation. I have a few questions though.

  1. So there is no need to manually copy the new binary to the location of the old binary? As you mentioned, renaming the original dockerd and setting the PATH variable to include the new binaries should do it.
  2. If I were to fall back to the original dockerd in /usr/bin , I should name the original binary to ‘dockerd’ again but what to do to the PATH variable that we modified to include the path to the new binary?

There shouldn’t be a need to copy the new binary to /usr/bin if you add the new binaries path to PATH variable.

Also, looking at the binaries again in /usr/bin , I realised you will have to backup other binaries too and not just dockerd. For me, i made modification that would just affect dockerd, so it worked out for me.

List of all binaries that need to be backed-up / renamed: containerd, containerd-shim, ctr, docker-init, docker-proxy, runc & dockerd.

What you could do is move all these to another folder. add bundles/binary-daemon to the $PATH to use new binaries.

if you wish to revert back to old binaries, just move the old binaries back to /usr/bin.
$PATH can be left as is, when adding new path to the end (PATH=$PATH:/new/path/). In this case binaries will be searched in paths that come before, and old dockerd will be able to locate required binaries in /usr/bin before ever reaching to bundles/binary-daemon path.

Don’t forget to follow the process to restart docker service every time you change binaries :slight_smile:

1 Like

I was able to successfully test the customized daemon in a Raspberry Pi. However I noticed that just adding the PATH didn’t work and I had to manually copy the binaries in /usr/bin/. Anyway this is a trivial issue for now. The changes I made were very minimal and only replacing dockerd worked fine. But with the same replaced binary when I issue docker swarm init, it gets stuck. The docker info command shows the Swarm status as pending. I suspected that other binaries must be also copied under /usr/bin but that made no difference.

docker swarm init worked fine with the original binaries from the docker installation so I suspect that with the replaced binary something has gone wrong but can’t figure out exactly what is the issue here.

I am able to solve this. It came out to be an error in the code changes I made. After fixing the bugs and creating the new binary, i’ m able to launch swarm.