Here is our setup:
We have a bunch of AWS EC2 instances that run our unit tests.
As long as the instance is up and running and has cached previous docker images, there is very little time in running the unit tests. On a fresh instance which has never run docker before, 99% of the time is spend downloading/building images.
Ideally we would like all the instances to mount /var/lib/docker to a common (S3?) shared location so that they don’t have to re-download the entire universe, but it doesn’t look like this is possible:
“Sorry, new users can only put 2 links in a post.” ==> link to stackoverflow page
So if sharing /var/lib/docker across hundreds of instances is not possible, how about having an EBS snapshot that has 90% of the images already downloaded, so that when the instance boots, it gets a copy of these images.
Is that possible? What’s the recommended way of sharing docker cache across multiple machines?
Basically, we spin up hundreds of EC2 instances a day, and 99% of the time those instances are just downloading/building images just to run our unit tests.
(We run our tests on spot instances, so we can’t “stop” these instances and restart them with the cache intact)