Accessing files from multi-container apps and using all files in a volume

General
1

Hi,

I am new to using Docker and would really appreciate any opinions on how to set this up/if Docker is appropriate for solving this problem. What I would like to do is to have the situation where multiple containers act as individual farms, that sell cows to a ‘market’ on a ‘market day’ as part of a larger daily ‘farm-life’ simulation. I have a simple python script that creates a .txt files of random sold animals, and these are written into a volume – and what I need is either another container/something else to do is keep track of how many ‘farms’ have written their files and sold the cows to the market, then once all farms have sold their selected cows to the market, send a file back to each farm to allow the simulation to continue with other parts. I also need a way for this to happen multiple times, so for example the simulation would run for 365 days and cows would be sold and bought from the volume every week, so the containers need wait for each other before progressing. I’d be grateful for any ideas on how to set this up/if it is possible, currently this code writes a file into the volume from each farm container using docker-compose up --build --scale farm_1=5.

Dockerfile

FROM python:3.8
RUN mkdir /app
ADD . /app
WORKDIR /app
ENTRYPOINT ["python3", "animals.py"]

market.Dockerfile

FROM python:3.8
RUN mkdir /app
ADD . /app
WORKDIR /app
CMD ["python3"]

Docker compose file

version: '2'
services:
  farm_1:
    build:
      dockerfile: Dockerfile
    ports:
    - "5000"
    volumes:
      - market_volume:/app/data/cows_to_sell
  the_market:
    build:
      dockerfile: market.Dockerfile
    ports:
        - "4000"
    volumes:
        - market_volume:/app/data/cows_to_sell
    depends_on:
        - farm_1
volumes:
  market_volume:

animals.py

import random
def generate_farm_name():
    farm_name = "Farm_" + str(random.choice(range(0,100)))
    return(farm_name)

farm_name = generate_farm_name()

market_name = './data/cows_to_sell/' + farm_name + '.txt'

f = open(market_name, "w+")

animals = ['cow A', 'cow B', 'cow C', 'cow D', 'cow E', 'cow F']

def write_animals(farm_name, market_name):
    selected = random.choice(animals)
    with open(market_name, 'a') as f:
        f.write(farm_name + " " + selected + "\n")


write_animals(farm_name, market_name)
2

You insist on txt files? You could use microservices, queues, a database etc.

You can bind mount a folder into every container of the service, you could use the random hostname as file extension, so every container creates their own file on the host. Then you can watch or periodically scan for response. Bind mount for easier access to the files for debug.

3

Thanks - no it doesn’t have to be .txt files, just an example, I’m open to writing into anything that will work, do you have any ideas how I would be able to set up the watching or scanning part?

4

Why not use web services and send messages between them, store in a common database?

1 Like