Running docker from within a chroot

Hey all,
I am trying to build a distro for RaspberryPi which ships with an aarch64 chroot running docker.

For this to work, I need docker to run within a chroot.
I am basing of rasbian, which is distributed only in arm32v7.

I have gotten as far as starting dockerd in the chroot using this command:

#!/usr/bin/env bash
/usr/bin/cgroupfs-mount
/usr/bin/dockerd -H unix://

However, when I try and bring up a container I get:

docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
256ab8fe8778: Pull complete 
Digest: sha256:d1668a9a1f5b42ed3f46b70b9cb7c88fd8bdc8a2d73509bb0041cf436018fbf5
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\"": unknown.
ERRO[0005] error waiting for container: context canceled 

This happens with any container.

Docker service is up, docker build gives similar issues;

Running in 74fe7f0514d0
OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\"": unknown

I assume this is happening because I am in a chroot. I have the following mount bindings:

# fstab: static file system information for chroots.
# Note that the mount point will be prefixed by the chroot path
# (CHROOT_PATH)
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/proc           /proc           none    rw,bind         0       0
/sys            /sys            none    rw,bind         0       0
/dev            /dev            none    rw,bind         0       0
/dev/pts        /dev/pts        none    rw,bind         0       0
/home           /home           none    rw,bind         0       0
/tmp            /tmp            none    rw,bind         0       0

# It may be desirable to have access to /run, especially if you wish
# to run additional services in the chroot.  However, note that this
# may potentially cause undesirable behaviour on upgrades, such as
# killing services on the host.
/run           /run            none    rw,bind         0       0
/run/lock      /run/lock       none    rw,bind         0       0
/dev/shm       /dev/shm        none    rw,bind         0       0
/run/shm       /run/shm        none    rw,bind         0       0

The sources of what I am writing are also available here:

Thanks,