Dockerize Mapr Cluster on a single server

General
, , , ,
1

Is it possible to run multiple containers in a single server and run a cluster?

2

Yes, its possible. I have run multiple clusters for testing purpose on multiple containers.

You can find some of my sample works here. Its mostly based on Cloudera.


3

I see so it is possible. Can you advice me on my analogy on making this possible? Here are the steps im playing on my mind.

First I need to download a base image (centos) then run it multiple times (3 containers)
Setup a network for those 3 to communicate
And that would end up as a cluster.

4

You are almost right.

From my experience this is what I do. I use ansible for creating Docker network and containers.

- name: Create Docker Network
      docker_network:
        name: "{{ network_name }}"

- name: Create Hadoop Docker Cluster
  docker_container:
    image: "{{ docker_image }}"
    name: node-{{ item }}
    hostname: node-{{ item }}
    network_mode: bridge
    networks:
      - name: "{{ network_name }}"
    purge_networks: true
    command: /bin/sleep infinity
    privileged: true
  with_sequence: count=6
  • Step 1 - Create a Docker user defined network
  • Step 2 - Spin up multiple containers attached to that network (this will help in inbuilt DNS resolution)

One points to note in step 2 is keep your container name and hostname same, and short name. Because docker internal DNS will use container_name.network_name as the fqdn. If you follow this the DNS resolution inside your containers will be perfect (which is very much required especially in a hadoop env).

Let me know how you go with this setup.

1 Like
5

Ok sure! , technically you are correct on steop 2 note with the name and hostname the same. Going to see this ansible for creating docker network and container never heard of it before

6

You could use Docker Compose for the same. I prefer Ansible just because its something which I use for automating infrastructure.