The concept of containers is more to separate the runtimes and resources of one machine into smaller chunks that do not interfere with each other.
I think what you’re looking for is the reverse in which case it’s a private cloud. However, I have not heard of one will merge CPU and GPU. You can use something like openstack.org and their “nova project” to manage compute resources but I never tried them myself.
Actually docker-swarm has the capability. Am looking into using docker-machine with the swarm to create a Virtualization layer across all workers. Just wanted to know if anyone has done this.
Docker swarm does not spread a single containers across the nodes, it may spread replicas but each replica stands on its own and is fronted by a virtual load balancer here’s how my 3 node swarm looks like
Each box is a separate container and they are not synchronized automatically even if it is replicated. Though my understanding of synchronize is that the memory and CPU between the containers are kept the same in which case this does not.