Buildx failing to cache from insecure registry

I have an insecure registry setup on a local network machine at registry_address:5000/repo. In my docker daemon configuration I have

  "insecure-registries": [
    "registry_address:5000"
  ]

If I run docker manifest inspect --insecure registry_address:5000/repo/image:tag, then the manifest information is displayed (without --insecure I see no such manifest: registry_address:5000/repo/image:tag)

If I try to build an image using buildx that tries to --cache-from the insecure registry

docker buildx build -t image:tag --pull --cache-from=type=registry,ref=registry_address:5000/repo/image:tag --cache-to=type=inline,mode=max --output=type=docker /path/to/Dockerfile

I see an error stating

ERROR importing cache manifest from registry_address:5000/repo/image:tag

I have tried creating a buildkitd.toml configuration file containing

[registry."registry_address:5000/repo"]
  insecure = true

I also tried it without the /repo, but with no difference.

and creating a buildx builder using this configuration file

docker buildx create --name buildx_builder --config buildkitd.toml && docker buildx use buildx_builder

However, this had no effect.

What do I need to do in order to have buildx recognise the cache manifest in my insecure registry?

The image in the insecure registry was built using the same command as above

The current setup I have has the insecure registry hosted on an Arch Linux machine and my local system is running Docker Desktop for Windows with WSL2

docker version on the server reports

$ docker version
Client:
 Version:           20.10.13
 API version:       1.41
 Go version:        go1.17.8
 Git commit:        a224086349
 Built:             Sat Mar 12 14:11:41 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.13
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.8
  Git commit:       906f57ff5b
  Built:            Sat Mar 12 14:10:25 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.1
  GitCommit:        10f428dac7cec44c864e1b830a4623af27a9fc70.m
 runc:
  Version:          1.1.0
  GitCommit:        v1.1.0-0-g067aaf85
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker version on my local system reports

$ docker version
Client: Docker Engine - Community
 Version:           20.10.13
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 10 14:07:51 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:43:56 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0