Combining your ideas
- cadvisor current container performance metrics
- node-exporter current node performance metrics
- prometheus historical data
In addition to using the ServiceUpdate endpoint which can be accessed via TLS socket (ideal) or /var/run/docker.sock (unsecure but simpler) I think one can create a service that checks prometheus for cadvisor/node-exporter data and based on some logic increase the number of replicas as needed.
Although this is already done and supported in k8s. It would be “cool” to have something like this in swarm. I’m not really a big proponent of auto-scaling unless it is a matter of dynamically provisioning another AWS instance and let it register into the swarm because you pay for the CPU/Memory by time anyway according to a comment in Server Fault.