How to move docker image to docker-compose (so I can add more ports)

andke · May 20, 2023, 7:49pm

some of my docker images are started like this;

$ cat docker-compose.yml 
version: '3.8'
services:
  mqtt:
    container_name: mqtt
    image: eclipse-mosquitto:2.0
    restart: unless-stopped
    volumes:
      - "/home/pi/mosquitto-data:/mosquitto"
    ports:
      - "1883:1883"
      - "9001:9001"
    command: "mosquitto -c /mosquitto-no-auth.conf"

  zigbee2mqtt:
    depends_on:
      - mqtt
    container_name: zigbee2mqtt
    image: koenkk/zigbee2mqtt
    restart: unless-stopped
    volumes:
      #- ./data:/app/data
      - /home/pi/zigbee2mqtt-data:/app/data
      - /run/udev:/run/udev:ro
    ports:
      # Frontend port
      - 8080:8080
    environment:
      - TZ=Europe/Berlin
    devices:
      # Make sure this matched your adapter location
      #- /dev/serial/by-id/usb-Texas_Instruments_TI_CC2531_USB_CDC___0X00124B0018ED3DDF-if00:/dev/ttyACM0
      - /dev/ttyUSB0:/dev/ttyUSB0

but I can’t find how nodered is started:

$ docker ps --all
CONTAINER ID   IMAGE                   COMMAND                  CREATED         STATUS                     PORTS                                                                                  NAMES
b9186fbb95b3   koenkk/zigbee2mqtt      "docker-entrypoint.s…"   4 months ago    Up 4 months                0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                                              zigbee2mqtt
d72af5dd46e8   eclipse-mosquitto:2.0   "/docker-entrypoint.…"   4 months ago    Up 4 months                0.0.0.0:1883->1883/tcp, :::1883->1883/tcp, 0.0.0.0:9001->9001/tcp, :::9001->9001/tcp   mqtt
dd6470bf5577   nodered/node-red        "./entrypoint.sh"        4 months ago    Up 4 months (healthy)      0.0.0.0:1880->1880/tcp, :::1880->1880/tcp                                              mynodered
1e17b73a227c   eclipse-mosquitto:2.0   "/docker-entrypoint.…"   18 months ago   Exited (0) 4 months ago                                                                                           myservice_mqtt_1
bf4209f4debd   hello-world             "/hello"                 18 months ago   Exited (0) 18 months ago

I’d like to move nodered to docker-compose in order to easily add SNMP ports like

-p 161:161/udp
-p 162:162/udp

I have no idea about how nodered is started now, with which parameters, and how to “migrate” it to docker-compose.

rimelek · May 20, 2023, 9:44pm

I am not sure I understand the question, but you can use docker container inspect mynodered to get the metadata of the container which shows you all the parameters like mounted folders or allowed capabilites and so on.

andke · May 20, 2023, 9:53pm

thank you, so this gives me:

 docker container inspect mynodered
[
    {
        "Id": "dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2",
        "Created": "2022-12-21T21:52:40.894954259Z",
        "Path": "./entrypoint.sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 22993,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-01-19T17:20:43.611978636Z",
            "FinishedAt": "2023-01-19T17:18:45.492706532Z",
            "Health": {
                "Status": "healthy",
                "FailingStreak": 0,
                "Log": [
                    {
                        "Start": "2023-05-20T23:49:29.168470751+02:00",
                        "End": "2023-05-20T23:49:29.70610141+02:00",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2023-05-20T23:49:59.72832206+02:00",
                        "End": "2023-05-20T23:50:00.292651946+02:00",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2023-05-20T23:50:30.313912296+02:00",
                        "End": "2023-05-20T23:50:30.835439157+02:00",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2023-05-20T23:51:00.864505979+02:00",
                        "End": "2023-05-20T23:51:01.421649658+02:00",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2023-05-20T23:51:31.447880805+02:00",
                        "End": "2023-05-20T23:51:31.983950103+02:00",
                        "ExitCode": 0,
                        "Output": ""
                    }
                ]
            }
        },
        "Image": "sha256:35f3062b67d1af0a3a6110270f7fd9ced7049ac61c0560e253b3e770a86a5587",
        "ResolvConfPath": "/var/lib/docker/containers/dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2/hostname",
        "HostsPath": "/var/lib/docker/containers/dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2/hosts",
        "LogPath": "/var/lib/docker/containers/dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2/dd6470bf5577324318d545c9fd1e2f82884c4bc8caf089cc71313129bb52bca2-json.log",
        "Name": "/mynodered",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "node_red_data:/data"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "1880/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "1880"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/06be145635f1a3c1eec3fdd853e29c0851fb1f33d09c9d00486bc5426d780ddc-init/diff:/var/lib/docker/overlay2/002356923a8d9ea9e1f44e94a6ee0492277be6ee25447aeb5b08eee759dc8043/diff:/var/lib/docker/overlay2/963b7120aed938cc0d781834d157859fe3ed4e986371661aba12fe72f24c5889/diff:/var/lib/docker/overlay2/696f0435d2322a839210a6cab9a7827b3b894080c8614f75d8f0184435395251/diff:/var/lib/docker/overlay2/bb0e044f55a04ef694cfbd59bc7d8e6db4369a7de9689a59a7f0b8397ca2b6f0/diff:/var/lib/docker/overlay2/35c4e557cfb99b36055f1e17707e3d094bc89282fcdd0a2ef16a9e694ba50602/diff:/var/lib/docker/overlay2/8a1f78c4e24ba37f8844e149ad47c623956ce63edce2f68fec9db6bb35d4d92c/diff:/var/lib/docker/overlay2/876236a87408cb06791fae7810aec73207727490de9ff1141387567d7e92cd98/diff:/var/lib/docker/overlay2/6a56d8aced850b93dc3f6fd17f9ef2c9831f546ca137489aa5ba0e19cd879311/diff:/var/lib/docker/overlay2/b07117a73c17d1cd412ad08c671eee1bbbd3be234f4c9872d75f0d472310cc1d/diff:/var/lib/docker/overlay2/83366201535e1e41c942e141356f8204d722558953392857ab8c349227f78c81/diff:/var/lib/docker/overlay2/e1c54f31f96c00bf68ff43770884f55b81b3315b1230fe82bd3cc85813f2bc92/diff:/var/lib/docker/overlay2/f09f9c100f8136e199360eb2d548f6f36c49e31b81b00c3fd2b8ac6c4d922328/diff:/var/lib/docker/overlay2/5795d2626113e4823faeabdde114b537563b3e5a4e8094b0109caf5e2253569c/diff:/var/lib/docker/overlay2/c6b7814c8b11c500d8d2548c3122144b49acb16a0034100cbc46811d8ab3e471/diff:/var/lib/docker/overlay2/6b9b3f13b5b1b1e94d30b968953b1adbf63eea988db9a7c18ce2bbac78e95b02/diff:/var/lib/docker/overlay2/e70c25172f76a7b295c859995199f9eaa11b1dc3dd8932ff560e13435926478d/diff:/var/lib/docker/overlay2/530d54415235ecc51d79425824d2da1ab91aecf02a05d636c114fac36434d809/diff",
                "MergedDir": "/var/lib/docker/overlay2/06be145635f1a3c1eec3fdd853e29c0851fb1f33d09c9d00486bc5426d780ddc/merged",
                "UpperDir": "/var/lib/docker/overlay2/06be145635f1a3c1eec3fdd853e29c0851fb1f33d09c9d00486bc5426d780ddc/diff",
                "WorkDir": "/var/lib/docker/overlay2/06be145635f1a3c1eec3fdd853e29c0851fb1f33d09c9d00486bc5426d780ddc/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "node_red_data",
                "Source": "/var/lib/docker/volumes/node_red_data/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "dd6470bf5577",
            "Domainname": "",
            "User": "node-red",
            "AttachStdin": true,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "1880/tcp": {}
            },
            "Tty": true,
            "OpenStdin": true,
            "StdinOnce": true,
            "Env": [
                "PATH=/usr/src/node-red/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "NODE_VERSION=16.16.0",
                "YARN_VERSION=1.22.19",
                "NODE_RED_VERSION=v3.0.2",
                "NODE_PATH=/usr/src/node-red/node_modules:/data/node_modules",
                "FLOWS=flows.json"
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "node /healthcheck.js"
                ]
            },
            "Image": "nodered/node-red",
            "Volumes": null,
            "WorkingDir": "/usr/src/node-red",
            "Entrypoint": [
                "./entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "authors": "Dave Conway-Jones, Nick O'Leary, James Thomas, Raymond Mouthaan",
                "org.label-schema.arch": "",
                "org.label-schema.build-date": "2022-08-04T13:19:23Z",
                "org.label-schema.description": "Low-code programming for event-driven applications.",
                "org.label-schema.docker.dockerfile": ".docker/Dockerfile.alpine",
                "org.label-schema.license": "Apache-2.0",
                "org.label-schema.name": "Node-RED",
                "org.label-schema.url": "https://nodered.org",
                "org.label-schema.vcs-ref": "",
                "org.label-schema.vcs-type": "Git",
                "org.label-schema.vcs-url": "https://github.com/node-red/node-red-docker",
                "org.label-schema.version": "3.0.2"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "d24f8c678b99f3e1acee65d725b6a2a6a0286d4049753ed1ac396aea1f3588fc",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "1880/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "1880"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "1880"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/d24f8c678b99",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "c4ef5f663db649849052ba05d05b81ea1d721860c1e23bde2527bf228cd9dfe4",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "2e968d4fee6d099229fd908a9718cd3ee27b09306afaccda3f091fe11dc8dd14",
                    "EndpointID": "c4ef5f663db649849052ba05d05b81ea1d721860c1e23bde2527bf228cd9dfe4",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }

And now I would like to add this docker to docker-compose.yml (and stop it from loading however it is loading today)

rimelek · May 20, 2023, 9:56pm

I have no idea what that means in this context. You have the metadata. You can use it to create a new compose file. Can you rephrase what you wrote?

andke · May 20, 2023, 10:12pm

Q1: This container is running now, somehow started on boot
I just assume that In order to add it to the docker-compose.yml file - I must stop it and prevent the current way it is started on boot?

Q2: When populating docker-compose yml with it - I need to add among other things “volumes” - which in the “container inspect” results looks like:
" “Volumes”: null,"

which I doubt it correct for the docker-compose.yml - I tried to google for some instructions on how to transform data from “docker container inspect mynodered” to a section in docker-compose.yml - which I failed to find answer to.

rimelek · May 20, 2023, 10:34pm

Well. You could not “move” the currently running container anywhere but create a new container, so you could keep it running and create a new container using compose, but if you need the same forwarded ports than yes, you need to stop the container.

That Volumes is the list of volumes defined in the Dockerfile. I know, it is confusing, but if you are scrolling up, you will find the “Mount” option which is what you need.

andke:

        "Mounts": [
            {
                "Type": "volume",
                "Name": "node_red_data",
                "Source": "/var/lib/docker/volumes/node_red_data/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],

andke · May 20, 2023, 10:42pm

Thank you - the currently running nodered contains lots of modules and configuration, so I’d rather copy/move/reuse it as-is.

this is so far my best guess at a docker-compose.yml file section

  nodered:
    services:
      node-red:
        container_name: nodered
        image: nodered/node-red
        restart: unless-stopped
        volumes:
          - "/var/lib/docker/volumes/node_red_data/_data"
        ports:
          - 1880:1880
            #for SNMP
          - 161:161/udp
          - 162:162/udp

    volumes:
      nodered: {}

…And even now I doubt that I got the volumes right.

rimelek · May 20, 2023, 10:59pm

If the modules and configs are on the volume, you can. Otherwise you can’t anf you need to copy everything out from the container before you remove it. You can use docker cp to copy files and folders out.

You are right that it’s not right. And I realized I was not entirely right before either, because the “Volumes” parameter shows anonymous volumes. It doesn’t matter where they were defined, but you can only define anonymous volumes in a Dockerfile. You had a named volume but you don’ need the Source path, you need the name of the volume. In order to use it, you need to make it external since it was not created by the compose project:

nodered:
   services:
      node-red:
        container_name: nodered
        image: nodered/node-red
        restart: unless-stopped
        volumes:
          - type: volume
            source: node_red_data
            target: /data
        ports:
          - 1880:1880
            #for SNMP
          - 161:161/udp
          - 162:162/udp

volumes:
  node_red_data:
    external: true

Eevrything is in the section I quoted in my previous post but you haven’t really used the name of the volume nor the the destination path. The syntax is a little different in the compose file but you can find the right syntax in the documentation and the compose spec

github.com

compose-spec/compose-spec/blob/master/spec.md

<!---
This file is generated, changes will be overwritten or CI specification check will fail
All changes should be done in the markdown files located in the spec directory
-->
# The Compose Specification
{:.no_toc}

* ToC
  {:toc}

# Status of this document

This document specifies the Compose file format used to define multi-containers applications. Distribution of this document is unlimited.

## Requirements and optional attributes

The Compose specification includes properties designed to target a local [OCI](https://opencontainers.org/) container runtime,
exposing Linux kernel specific configuration options, but also some Windows container specific properties. It is also designed for cloud platform features related to resource placement on a cluster, replicated application distribution, and scalability.

We acknowledge that no Compose implementation is expected to support all attributes, and that support for some properties

This file has been truncated. show original

I used the long syntax of the volume definition so you can see how similar it is to the values in the metadata json, but you could use the short syntax as well, it doesn’t matter.

volumes:
  - node_red_data:/data

Docker will find the source path by name and mount it.

andke · May 21, 2023, 8:13am

Thank you. I’ll make backups of all and the partition before proceeding.
It’s ironic, but I do it mostly because I failed to find information on how to add extra ports to this running container (-p 161:161/udp
-p 162:162/udp)

but moving to docker-compose will make things easier in the future

rimelek · May 21, 2023, 8:25am

Docker compose will do the same. Stop the container and start a new container with different ports. It is just quicker. That’s why you haven’t found anything about adding more ports because it is not possible without stopping the container. Without that you could only configure a reverse proxy and forward requests to the containe’s IP address.

meyay · May 21, 2023, 9:11am

Even though the question of how to create a compose file from a running container is already sorted out, I want mention a lazier approach: https://github.com/Red5d/docker-autocompose will create a compose file with all properties from docker inspect of a running container. Of course, it needs some clean up afterward, but it could still be a good starating point.