Updating image of elk using docker

General Discussions
1

i am using docker-compose for running elk services and i have kafka and zookeeper in the same docker-compose file i want to upgrade the services of elasticsearch kibana and logstash without interrupting the kafka and zookeeper
2

just update the image tags and perform docker-compose up -d? The command will detect which services change and only deploy those. This is at least the behavior if all images are referencing repos and not beeing build within the docker-compose.yml. Not sure if the behavior is the same if you actualy do build images within the docker-compose.yml.

3 

version: "3.3"
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: elasticsearch
    environment:
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
    secrets:
      - source: elasticsearch.yml
        target: /usr/share/elasticsearch/config/elasticsearch.yml
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nproc: 20480
      nofile:
        soft: 160000
        hard: 160000
    logging:
      driver: "json-file"
      options:
        max-file: "9"
        max-size: "6m"
    restart: always
    ports:
      - 9200:9200
    networks:
      - esnet
  kibana:
    image: docker.elastic.co/kibana/kibana:7.6.0
    container_name: kibana
    depends_on:
      - elasticsearch
    restart: always
    logging:
      driver: "json-file"
      options:
        max-file: "9"
        max-size: "6m"
    secrets:
      - source: kibana.yml
        target: /usr/share/kibana/config/kibana.yml
    networks:
      - esnet
  logstash:
    image: docker.elastic.co/logstash/logstash:7.6.0
    container_name: logstash
    volumes:
      - ./logstash/pipeline:/usr/share/logstash/pipeline
      - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./logstash/config/jvm.options:/usr/share/logstash/config/jvm.options
      - ./logstash/plugins:/usr/share/logstash/plugins
    restart: always
    logging:
      driver: "json-file"
      options:
        max-file: "9"
        max-size: "6m"
    networks:
      - esnet
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_DATA_DIR: /tmp/zookeeper
    logging:
      driver: "json-file"
      options:
        max-file: "9"
        max-size: "6m"
    restart: always
    ports:
      - "2181:2181"
    networks:
      - esnet
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 1
      KAFKA_CREATE_TOPICS: "filebeat:1:1,winlogbeat:1:1,packetbeat:1:1"
      KAFKA_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.0.133:9092
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'
      KAFKA_UNCLEAN_LEADER_ELECTION_ENABLE: 'true'
      KAFKA_DELETE_TOPIC_ENABLE: 'true'
      KAFKA_HEAP_OPTS: -Xmx1G -Xms1G
    logging:
      driver: "json-file"
      options:
        max-file: "9"
        max-size: "6m"
    restart: always
    ports:
      - '9092:9092'
    networks:
      - kafka
    depends_on:
      - zookeeper
    networks:
      - esnet
volumes:
  elasticsearch:
    driver: local
secrets:
  elasticsearch.yml:
    file: /home/aniket/docker/elasticsearch/config/elasticsearch.yml
  kibana.yml:
    file: /home/aniket/docker/kibana/config/kibana.yml
networks:
  esnet:

This is my full docker-compose file i just want to change in elasticsearch kibana and logstash
i want that if i run docker-compose up it will up all the container and i just want to up elasticsearch kibana and logstash i am not touching kafka zookeeper services

4

My experience with docker-compose is that if services are already deployed, only services with modified declaration get re-deployed when docker-compose up -d is executed. Though, if you deployed your stack as swarm stack (docker stack deploy), then all services will be re-deployed.

You could always create a small test compose.yml and actualy observe the update behavior yourself… It’s not rocket science.

I find it odd to see unrelated services mixed in a compose.yml While ELK is clearly aimed forward log management, zk+kafka are not. They belong in their own compose.yml.