i am using docker-compose for running elk services and i have kafka and zookeeper in the same docker-compose file i want to upgrade the services of elasticsearch kibana and logstash without interrupting the kafka and zookeeper
just update the image tags and perform
docker-compose up -d? The command will detect which services change and only deploy those. This is at least the behavior if all images are referencing repos and not beeing build within the docker-compose.yml. Not sure if the behavior is the same if you actualy do build images within the docker-compose.yml.
version: "3.3" services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0 container_name: elasticsearch environment: - "ES_JAVA_OPTS=-Xms1g -Xmx1g" volumes: - elasticsearch:/usr/share/elasticsearch/data secrets: - source: elasticsearch.yml target: /usr/share/elasticsearch/config/elasticsearch.yml ulimits: memlock: soft: -1 hard: -1 nproc: 20480 nofile: soft: 160000 hard: 160000 logging: driver: "json-file" options: max-file: "9" max-size: "6m" restart: always ports: - 9200:9200 networks: - esnet kibana: image: docker.elastic.co/kibana/kibana:7.6.0 container_name: kibana depends_on: - elasticsearch restart: always logging: driver: "json-file" options: max-file: "9" max-size: "6m" secrets: - source: kibana.yml target: /usr/share/kibana/config/kibana.yml networks: - esnet logstash: image: docker.elastic.co/logstash/logstash:7.6.0 container_name: logstash volumes: - ./logstash/pipeline:/usr/share/logstash/pipeline - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml - ./logstash/config/jvm.options:/usr/share/logstash/config/jvm.options - ./logstash/plugins:/usr/share/logstash/plugins restart: always logging: driver: "json-file" options: max-file: "9" max-size: "6m" networks: - esnet zookeeper: image: wurstmeister/zookeeper container_name: zookeeper environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_DATA_DIR: /tmp/zookeeper logging: driver: "json-file" options: max-file: "9" max-size: "6m" restart: always ports: - "2181:2181" networks: - esnet kafka: image: wurstmeister/kafka container_name: kafka environment: KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 1 KAFKA_CREATE_TOPICS: "filebeat:1:1,winlogbeat:1:1,packetbeat:1:1" KAFKA_LISTENERS: PLAINTEXT://kafka:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.0.133:9092 KAFKA_ADVERTISED_HOST_NAME: kafka KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false' KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE: 'true' KAFKA_DELETE_TOPIC_ENABLE: 'true' KAFKA_HEAP_OPTS: -Xmx1G -Xms1G logging: driver: "json-file" options: max-file: "9" max-size: "6m" restart: always ports: - '9092:9092' networks: - kafka depends_on: - zookeeper networks: - esnet volumes: elasticsearch: driver: local secrets: elasticsearch.yml: file: /home/aniket/docker/elasticsearch/config/elasticsearch.yml kibana.yml: file: /home/aniket/docker/kibana/config/kibana.yml networks: esnet:
This is my full docker-compose file i just want to change in elasticsearch kibana and logstash
i want that if i run docker-compose up it will up all the container and i just want to up elasticsearch kibana and logstash i am not touching kafka zookeeper services
My experience with docker-compose is that if services are already deployed, only services with modified declaration get re-deployed when
docker-compose up -d is executed. Though, if you deployed your stack as swarm stack (
docker stack deploy), then all services will be re-deployed.
You could always create a small test compose.yml and actualy observe the update behavior yourself… It’s not rocket science.
I find it odd to see unrelated services mixed in a compose.yml While ELK is clearly aimed forward log management, zk+kafka are not. They belong in their own compose.yml.