Hi everyone,
I have an elastic project running on a swarm cluster and I’m trying to share a file with a container so that I can update the file on the host and the change is automatically reflected on the container. The problem is every time I update the file on the host I need to remove the stack and the config file and deploy everything again.
I’m using compose version 3 to deploy everything more easily.
This is an example of my configuration for that file.
version: ‘3.3’
type: bind
configs:
Can anyone help me find a way to do this?
Tank you,
meyay
April 9, 2020, 4:35pm
2
This is normal behavior. A config name is distributed amongst the nodes during stack deployment. To archive a distribution of a changed config, you need to add a suffix to the config name (like the .v1 I added). Whenever you make a change to the file, you have to modify the suffix as well. As a result the “new” config will be distribution and applied to the container. On a stack delete, you will notice that all versions changed during uptime will be listed as delted as well…
version: '3.3'
...
configs:
- source: monitors_config.v1
target: /usr/share/heartbeat/monitors.d/monitor.icmp.yml
...
configs:
monitors_config.v1:
file: ./heartbeat/monitors.d/monitor.icmp.yml
HI Meyay,
Thank you for responding to my question so quickly.
The way you show me in the previous post, i still need to remove and deploy the stack or I just need to update the elastic_heartbeat service ?
Best regards,
meyay
April 11, 2020, 6:28pm
4
redeploy without removing it first.
I deployed the stack without removing it first, but now I have an error saying that "“No such image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0 ” in all containers.
meyay
April 11, 2020, 7:27pm
6
Seems like you changed more than just the handle on the config file and its content.
Since there is no before/after post of your docker-compose.yml, it is hard to guess what’s wrong.
I assume you use a recent version of the docker engine…
This is my docker compose file at this moment…
services:
elasticsearch:image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
kibana: image: docker.elastic.co/kibana/kibana:7.6.0
logstash: image: docker.elastic.co/logstash/logstash:7.6.0
heartbeat: image: docker.elastic.co/beats/heartbeat:7.6.0
filebeat: image: docker.elastic.co/beats/filebeat:7.6.0{ELASTICSEARCH_USERNAME:******}
- ELASTICSEARCH_PASSWORD= {ELASTICSEARCH_PASSWORD:******}
metricbeat: image: docker.elastic.co/beats/metricbeat:7.6.0
configs:
networks:
The only thing that I’ve change was the suffix to version 2
I’am using this command to update the stack docker stack deploy - c docker-compose.yml elastic
And hove everything works .
meyay
April 11, 2020, 7:51pm
8
The docker-compose.yml is hard to read like this. You need to add a blank line before the preformated text block to actualy apply the formating.
I used this pattern a lot in the past and never had problems. What happens if you execute docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.0?
I assume your added a space between the dash and the c in “-c” by accident.
version: ‘3.3’
services:
elasticsearch:docker.elastic.co/elasticsearch/elasticsearch:7.6.0
type: bind
“9200:9200”
“9300:9300”
elknet
kibana:docker.elastic.co/kibana/kibana:7.6.0
“5601:5601”
type: bind
elknet
logstash:docker.elastic.co/logstash/logstash:7.6.0
“5000:5000”
“9600:9600”
type: bind
type: bind
elknet
heartbeat:docker.elastic.co/beats/heartbeat:7.6.0
type: volume
type: bind
output.elasticsearch.hosts=[“elasticsearch:9200”]
elknet
filebeat:docker.elastic.co/beats/filebeat:7.6.0
“9002:9002”
type: bind
/var/run/docker.sock:/var/run/docker.sock:ro
/var/lib/docker/containers/:/var/lib/docker/containers/:ro
ELASTICSEARCH_USERNAME={ELASTICSEARCH_USERNAME:******}
ELASTICSEARCH_PASSWORD={ELASTICSEARCH_PASSWORD:******}
elknet
metricbeat:docker.elastic.co/beats/metricbeat:7.6.0
type: bind
/var/run/docker.sock:/var/run/docker.sock:ro
/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
/proc:/hostfs/proc:ro
/:/hostfs:ro
elknet
configs:
networks:
Yes the - c was a typo.
meyay
April 11, 2020, 8:16pm
10
The indention is still broken on your pasted compose.yml.
juts for the sake of demonstration… your preformated text should look like this:
version: '3.3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
environment:
ELASTIC_PASSWORD: ******
discovery.type: single-node
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
configs:
type: bind
source: elastic_config
target: /usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "9200:9200"
- "9300:9300"
networks:
elknet:
...
Paste your compose.yml, then mark the text, then click on “preformated text”.
It’s given me an error about symlink broken.
The worker node only has 4Gb free of disk space I’m going to do some cleaning.
version: '3.3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
environment:
ELASTIC_PASSWORD: changeme
discovery.type: single-node
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
configs:
- type: bind
source: elastic_config
target: /usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "9200:9200"
- "9300:9300"
networks:
- elknet
kibana:
image: docker.elastic.co/kibana/kibana:7.6.0
ports:
- "5601:5601"
configs:
- type: bind
source: kibana_config
target: /usr/share/kibana/config/kibana.yml
networks:
- elknet
logstash:
image: docker.elastic.co/logstash/logstash:7.6.0
ports:
- "5000:5000"
- "9600:9600"
configs:
- type: bind
source: logstash_config
target: /usr/share/logstash/config/logstash.yml
- type: bind
source: logstash_pipeline
target: /usr/share/logstash/pipeline/logstash.conf
environment:
LS_JAVA_OPTS: -Xmx256m -Xms256m
networks:
- elknet
heartbeat:
image: docker.elastic.co/beats/heartbeat:7.6.0
configs:
- type: volume
source: heartbeat_config
target: /usr/share/heartbeat/heartbeat.yml
read_only: true
- type: bind
source: monitors_config.v2
target: /usr/share/heartbeat/monitors.d/monitor.icmp.yml
environment:
- output.elasticsearch.hosts=["elasticsearch:9200"]
networks:
- elknet
filebeat:
image: docker.elastic.co/beats/filebeat:7.6.0
user: root
ports:
- "9002:9002"
configs:
- type: bind
source: filebeat_config
target: /usr/share/filebeat/filebeat.yml
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /var/lib/docker/containers/:/var/lib/docker/containers/:ro
command: ["--strict.perms=false"]
environment:
- ELASTICSEARCH_USERNAME=${ELASTICSEARCH_USERNAME:$elastic}
- ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD:$changeme}
networks:
- elknet
metricbeat:
image: docker.elastic.co/beats/metricbeat:7.6.0
user: root
configs:
- type: bind
source: metricbeat_config
target: /usr/share/metricbeat/metricbeat.yml
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
- /proc:/hostfs/proc:ro
- /:/hostfs:ro
networks:
- elknet
configs:
elastic_config:
file: ./elasticsearch/config/elasticsearch.yml
logstash_config:
file: ./logstash/config/logstash.yml
logstash_pipeline:
file: ./logstash/pipeline/logstash.conf
kibana_config:
file: ./kibana/config/kibana.yml
heartbeat_config:
file: ./heartbeat/config/heartbeat.yml
filebeat_config:
file: ./filebeat/config/filebeat.yml
metricbeat_config:
file: ./metricbeat/config/metricbeat.yml
monitors_config.v2:
file: ./heartbeat/monitors.d/monitor.icmp.yml
networks:
elknet:
driver: overlay
meyay
April 11, 2020, 9:13pm
13
You compose.yml looks fine. From my pov it seems impossible anything in the pasted compose.yml should cause the image pull problem. Renaming the config handles and experience pull problems afterwards are accidentally causal at best.
Please loose the “type: bind” when you use a config with a service, There is a reason why my first example didn’t include this setting: it does not exist (see: docs )
Thank you for the input, I’ve done the correction as suggested I’m gonna try bringing the stack up again and I will update this post with the result
meyay
April 11, 2020, 9:36pm
15
Oh, and neither “type: volume”, nor “read_only: true” exist for configs. You seem to mixed up the attributes with those for volumes.
I will leave you up to yourself now. The original question is answered. Though, it seems you have a different problem. Good luck!
Thanks meyay! I just got a solution from you!
Regard: CEO of Account Generator
Shared hosting file system works!10freeaccounts
