Java Rest Servers in many dockers - Slow starting time

Hello everyone,

I have developed an application to simulate many computers each running a java REST server using docker.
Java Rest server is a Embedded Tomcat REST Server (I am not exactly aware as my colleague developped this part)

I use linux to spawn my containers. I can spawn multiple containers, each with their own java REST server running.

However, although the java application starts correctly (visible using its log) on each container, it takes a long time for the REST interface to be accessible using for example a simple CURL command.
With 4 containers, it takes up to 60 seconds for all the REST interfaces of the different containers to be accessible. (Sometimes it only takes a couple of seconds and sometime much more …)
With 10 containers, it tools about 10 minutes
With 60 containers, I stopped waiting …

Anyone experienced a similar issue or has any input on my problem ? Any link or reading material will be largely appreciated.
Could I see how much computing time each container get ? Could I force a container to have some computing time ?

Thanks,
Simon

What version of docker are you running and on what OS? Are you using a mounted volume?

Dear mikeball,

I am running docker on Linux (Ubuntu 16.04.1 LTS)
It is Docker version 1.12.1,
I don’t think I am mounting volumes.

I am generating the containers using the following image:
FROM ubuntu:14.04
MAINTAINER <simon ruffieux simon.ruffieux@gmail.com>

RUN apt-get update 
RUN apt-get install -y iperf olsrd 
RUN apt-get install -y git-core 
RUN apt-get install -y make
RUN  apt-get update 
RUN apt-get install -y build-essential  gcc

## ADDED SR - INSTALL EXTRA PACKAGES on TAKE_VMs 
RUN apt-get install -y curl
RUN apt-get install -y python python-dev python-setuptools python-netifaces 
RUN apt-get install -y python-pip
RUN pip install APScheduler
RUN apt-get install -y traceroute
## Needed to use add-apt-repository
RUN apt-get install -y software-properties-common
RUN apt-get update -y 
RUN add-apt-repository ppa:openjdk-r/ppa
RUN apt-get update -y
RUN apt-get install -y openjdk-8-jdk

ADD files/dockemu-launch.sh /
RUN chmod +x /dockemu-launch.sh
ENTRYPOINT [ "/dockemu-launch.sh" ]

The dockemu-launch script is used to start olsrd on system start and then wait until eth0 interface is ready

To provide you with a better description of the intended goal of the system:

I have a host machine (Linux) which simulates a network using NS-3, generates N containers and connect them to that simulated network using tap bridges. So each container has ts own IP address linked to that NS-3 network.

On each linux container, I have a javaRest server which automatically sends messages to the javaRest server located on the other containers (the application knows their IP). Note that each container also runs olsrd to generate/manage the routing table.

The idea is to be able to generate different topology for the network and investigate the effect on RTT, etc.The idea is also to be able to better handle message distribution from the java application (retransmission, multiple broadcast, etc)