Docker Swarm service exposed port can't be accessed over host's IPv6

I’m trying to setup gitlab with docker swarm, and access it over IPv6. Here’s my compose file:

services:
  gitlab:
    image: gitlab/gitlab-ee:latest
    ports:
      - "2222:22"
      - "80:80"
    volumes:
      - gitlab-config:/etc/gitlab
      - gitlab-logs:/var/log/gitlab
      - gitlab-data:/var/opt/gitlab
      - type: tmpfs
        target: /dev/shm
        tmpfs:
          size: 256000000
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://git.box.ygg'
        gitlab_rails['gitlab_shell_ssh_port'] = 2222
        gitlab_sshd['enable'] = true
        gitlab_sshd['listen_address'] = ':2222'
    networks:
      - box-network

volumes:
  gitlab-config:
  gitlab-logs:
  gitlab-data:
  gitlab-runner-config:

networks:
  box-network:
    driver: overlay
    attachable: true
    driver_opts:
      com.docker.network.driver.mtu: 1280

git.box.ygg is an entry in /etc/hosts pointing to the devices IPv6 address.

When I try connecting to that SSH port 2222 over IPv6, nothing happens, but it works if I use the IPv4 address. The same happened to the HTTP 80 port, but I solved that with an NGINX proxy on the host system.

I found some old issues which mention that IPv6 is not supported, but then it said that it should be now - enable_ipv6 not recognised from version 3.0 · Issue #4958 · docker/compose · GitHub, and looking at the docker network, enable IPv6 is set to false:

$ docker network inspect box_box-network
[
    {
        "Name": "box_box-network",
        "Id": "v5tfhjzfgdhfbrmcpo1qxujsg",
        "Created": "2024-07-11T22:52:56.996325871Z",
        "Scope": "swarm",
        "Driver": "overlay",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "10.0.1.0/24",
                    "Gateway": "10.0.1.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "0549df91712e9cca237f1ca4b41282c1ff63b2b3d4e028d4c5b5b6ca7a3da0ff": {
                "Name": "box_gitlab.1.zceqci4t1mubuzaetjscvkpt5",
                "EndpointID": "e81e1bdf64a770b8e4c329ecda489246a5e2d3b81744183be9015e38531f1395",
                "MacAddress": "02:42:0a:00:01:49",
                "IPv4Address": "10.0.1.73/24",
                "IPv6Address": ""
            }
...

And setting enable_ipv6: true and the range in ipam options on the network in compose says that’s enable_ipv6 is not allowed.

Running this version on arch linux:

$ docker -v
Docker version 27.0.3, build 7d4bcd863a

Any ideas or hints? Thanks.