I have a VM which runs a local Docker Hub (in a container), a Gitlab Runner (in a Container) and a Apache Spark Cluster orchestrate witch Docker Swarm.
So my plan is to create a Gitlab CI/CD pipeline, where i can deploy new Spark worker (as Container inside the Swarm) using a deploy stage in Gitlab CI/CD. Gitlab itself does not run on that same VM.
My problem is, i dont know how i get a Connection from the CI/CD to the Swarm. Even if they run on the same VM and use the same Docker Engine, i think i need to handle the Swarm cluster as a remote cluster.
I hope someone can give me a hint.