Update: I have found a solution. Apparently when creating a swarm of masters the majority of (n+1)/2 must be present and active for swarm to function. So if there are 2 managers and one goes down, swarm goes down. However if there are 3 managers and one goes down the other two keep working. The initial master can be started with:
docker swarm init --listen-addr $(hostname -i):2377 --auto-accept manager --auto-accept worker
Then the other managers can be started with:
docker swarm join --manager --listen-addr $(hostname -i):2377 $FIRST_MANAGER_IP:2377