VPC Peering Docker-VPC to Default-VPC

General
1

Expected behavior

I expected to be able to access my RDS / ElastiCache from within the Docker Swarm after updating SecurityGroup rules

Actual behavior

SecurityGroup rules could not be created, as there is no VPC peering between Docker-VPC and default-VPC

Additional Information

It would be cool (if possible) if that VPN Peering and Routing tables were created automatically, especially as the Routing-Table for Docker-VPC is created by CloudFormation and is likely to be overwritten on upgrade…

If that’s not possible for whatever reasons, it would be nice to have the solution (create VPC Peering, update Routing-Tables) documented.

Steps to reproduce the behavior

  1. Create any resource (RDS, ElastiCache, anything) in VPC
  2. create swarm
  3. try to start a container that needs resource from step 1 as a service
2

Thanks - we know that VPC inter-op will be important. Peering is one option, but I’d be interested on your thoughts on whether installing Docker for AWS into your existing VPC (with the database) would be a good option?

You could probably set up the peering manually, and then be careful when applying Cloudformation stack updates (which is a manual process initiated by you anyway).

If it’s ok to expose, you can consume the database using its public address as a workaround: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.Scenarios.html#USER_VPC.Scenario3

Let us know if you have any other feedback.

3

wouldn’t have a problem with putting the docker stack into an existing VPC.

I did set up the peering manually already. And since the aws-stack is beta, I was planning to create a new stack from scratch on update, then switch Route53 anyways. Probably try the stack-update afterwards before I tear it down to see if my fears were uncalled for :wink:

I don’t really want to make the database and cache open for the world to access.

4

I’ve actually been waiting for the ability to install in an existing VPC, specifically for reasons like this.

5

Hey @friism has docker made a decision on this? I’ve setup VPC Peering for the moment. I could go either way with our direction. Something that needs to be added is the ability to set the VPC CIDR as a formation parameter. I got lucky that I don’t have an overlap otherwise the VPC Peering would have not worked with our existing configuration without modification to the template.

I think it would simplify the installation using an existing VPC, however I also kinda dig the isolation, but configuration regardless I think is the best option. Just my 2 cents.

-Adam

6

We’ve almost completed work to make Docker for AWS installable into existing VPC - stay tuned for the next beta!

1 Like
7

I just kicked the tires with the new Swarm mode in Docker Cloud and this is the first thing I noticed: no ability to launch the swarm into an existing VPC.

This will be a requirement for us to use Docker for AWS, in our case Swarm through Docker Cloud. So hopefully this comes out soon in Docker Cloud as well.

8

Docker for AWS now has the option of installing into an existing VPC: https://docs.docker.com/docker-for-aws/#install-with-an-existing-vpc

9

For now you can try the Docker for AWS template that supports existing VPCs and then registering the newly created Swarm with Docker Cloud for fleet management. Docker Cloud will support this in the future.

10

Good point. Especially since Docker Cloud doesn’t know how to terminate and clean up the Swarm for me, I’m not sure there is any benefit of having Docker Cloud create the swarm in the first place.

[Returns to AWS console to find all the ELBs and Security Groups and Instances needing to be deleted.] :frowning:

11

I’d suggest you use cloudformation to delete the stack created by Docker for AWS (through Docker Cloud). That’ll get rid of all the resources created.

12

Deleting through CloudFormation was, indeed, the right choice. Thanks for the tip.