Upgrade ELK on docker from 7.10 to 7.17

General
, ,
1

How to upgrade Elasticsearch , Kibana installed as 3 nodes cluster with docker compose from 7.10 to 7.17 ? This link shares other methods but not containers installed/started with docker compose - swarm.
Is their a step by step documentation for the same?

Setup as below:
3-node ElasticSearch cluster (each node on respective server) and a single Kibana instance installed using docker-compose.

Attached es01-docker-compose.yml is for one of the node. similar files goes for other nodes.
es01-docker-compose.txt (1.9 KB)

2

Please do not attach compose files as file attachments, but instead add their content within a code block (</> icon), so everyone can immediatly see the content.

Furthermore, can you explain how this post is related to swarm stack deployments? You compose file version indicates a single node docker compose deployment, and you write that you use docker-compose for deployment.

Also the use of variables in the compose file indicate there must be an .env file in the folder where the compose file is stored. To change the image tag, you need to change the value of the VERSION variable in the .env file and perform a docker-compose up -d or docker compose up -dif you installed the compose cli-plugin.

3

Please see the description 'Attached es01-docker-compose.yml is for one of the node. similar files goes for other nodes.

Yes, there is .env file where version no. is mentioned.

I changed the version no. While bringing up docker-compose up, after installion, gets below error. The containers were stopped.

"Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.17.6
Creating es03 … error
Creating es01 …
Creating kib01 …
Creating es02 …

Creating es02 … error
ner to be able to reuse that name.

ERROR: for es02 Cannot create container for service es02: Conflict. The container name “/es02” is already in use by container “”. You have to remove (or rename) that contaiCreating kib01 … error
Creating es01 … error
ERROR: for kib01 Cannot create container for service kibana: Conflict. The container name “/kib01” is already in use by container “”. You have to remove (or rename) that container to be able to reuse that name.

ERROR: for es01 Cannot create container for service es01: Conflict. The container name “/es01” is already in use by container “”. You have to remove (or rename) that container to be able to reuse that name.

ERROR: for es03 Cannot create container for service es03: Conflict. The container name “/es03” is already in use by container “”. You have to remove (or rename) that container to be able to reuse that name.

4

That’s not realy the answer to my question. Though, I understand: it is not related to swarm at all. Those are just three independend docker-compose deployment on three different nodes.

According the rest of your post: it doesn’t seem you found the correct compose file OR did not specify the correct project name (which by default is the folder name).

5

My bad! You are correct. The attached is not the correct compose file. Please have it removed. There are 3 nodes in same compose as below… But for the error shared: Is it advisable to remove the previous containers es01,es02,es03, kib01 and then perform docker compose up?
Also, if configuration is as per the attached compose file i.e. three independend docker-compose deployment on three different nodes, does the upgrade process differ?

version: '2.2'
services:
  es01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.6
    container_name: es01
    environment:
      - node.name=es01
      - cluster.name=es-docker-cluster
      - discovery.seed_hosts=es02,es03
      - cluster.initial_master_nodes=es01,es02,es03
      - bootstrap.memory_lock=true
      - reindex.remote.whitelist=<servername>:9206
      - "ES_JAVA_OPTS=-Xms1024m -Xmx1024m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9216:9200
    networks:
      - elastic
  es02:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.6
    container_name: es02
    environment:
      - node.name=es02
6

I would strongy advise to find out what the reason is that the containers are not detected as beeing the same containers of the same compose project - they should!

You can inspect one of the involved containers and take a look at the labels to see which project name was used for deployment and then use the very same project name for your deployment as well.

docker compose will detect changes in a service for the same project name and then stop the old container, remove it and create a new one based on the new image by itself. Though, if you change the service name, the project name (e.g. by copying the compose file to a different folder) then this will not be an in place update.

7

In another environment, docker-compose.yml image mentions the version as below and there is no .env file as well. Where should i update the version number in such a case to upgrade ES?

image: docker.elastic.co/elasticsearch/elasticsearch:${VERSION}

8

I assume this is for company use. If this is the case then there should be a runbook about how things need to be done. Please look at it. If it doesn’t exist, get the person who did the last deployment and ask them for directions.

Note: a tag is either hard coded and needs to be changed direct in the compose file, or is a variable that need to be changed in the env file. Though, you can always replace a variable with a hard-coded value. The default env file is .env in the same folder as the compose file, but it could also be overriden by the --env-file argument.

You could check the bash history for evidence wether the env-file argument is used.