Issues getting Volumes to create when launching compose file

General
1

Expected behavior

When launching compose.yaml Azure is not creating the volumes. When I run in Docker it is creating the volumes necessary for my expected outcome.

Actual behavior

No Volumes are being created.

Additional Information

Testing an EdFi year-specific database, I have made generic configurations to test in my personal account on Azure. I can get the file to the ACR and launch it, however, as stated the volume stores don’t create. I can link them by creating a separate Postgres DB, but I would like to see the persistent volumes create.

Also, when I push to my subscription from the Docker desktop I am unable to get the volumes as well. Just trying to determine if I am missing something or if Azure blocks this and forces you to use an online storage service in your Azure cloud. Ultimately, I suppose it doesn’t matter but as a stubborn goat, I want to try.

I have no issue reproducing the error in Azure Containers. I have tried both web apps container and container svc. And to be clear, yes I am new to this.

2

Do you mean you run the container using docker run ...? If you do, have you tried docker compose on your machine or in a simple virtual machine without Azure?

If cou can share your docker compose file or create one for us that has the same result without creating a volume defined in your compose file, we can help easier.

Update:

@rmassey03 I edited your reply to this post and the spam filter hid it. It will be restored soon so we can see your compose file, but please, use code blocks when you insert code because it was unreadable.

1 Like
3

Thank you for your reply. The command I am using is:

docker compose -f ./compose/pgsql/compose-year-specific-env.example.yml --env-file ./.env up -d

When I run the file locally in and in an VM. They work properly. I suspect the issue is that Azure prefers to use a service instead of a container volume. Again, I am new to this and Containers, I could be completely wrong. But from my reading in Linux forums, there are others that were successful in their builds.

# SPDX-License-Identifier: Apache-2.0
# Licensed to the Ed-Fi Alliance under one or more agreements.
# The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
# See the LICENSE and NOTICES files in the project root for more information.

version: "3.8"

services:
  db-admin:
    build:
      context: ../../DB-Admin/Alpine/pgsql
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: "${POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      API_MODE: YearSpecific
    volumes:
      - vol-db-admin:/var/lib/postgresql/data
    restart: always
    container_name: ed-fi-db-admin

  nginx:
    build:
      context: ../../Web-Gateway/Alpine
      dockerfile: Dockerfile
    environment:
      ADMINAPP_VIRTUAL_NAME: "${ADMINAPP_VIRTUAL_NAME:-adminapp}"
      ODS_VIRTUAL_NAME: "${ODS_VIRTUAL_NAME:-api}"
    ports:
      - "443:443"
      - "80:80"
    container_name: ed-fi-gateway
    restart: always
    hostname: nginx
    volumes:
      - ../../ssl:/ssl/
    depends_on:
      - api
      - adminapp

  api:
    build:
      context: ../../Web-Ods-Api/Alpine/pgsql
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: "${POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      POSTGRES_PORT: "${PGBOUNCER_LISTEN_PORT:-6432}"
      ODS_POSTGRES_HOST: EdFi_{0}
      ADMIN_POSTGRES_HOST: pb-admin
      API_MODE: YearSpecific
      ApiSettings__PathBase: "${ODS_VIRTUAL_NAME:-api}"
      TPDM_ENABLED: "${TPDM_ENABLED:-true}"
      ODS_WAIT_POSTGRES_HOSTS: "EdFi_Ods_2021 EdFi_Ods_2022 "
    volumes:
      - ${LOGS_FOLDER}:/app/logs
    depends_on:
      - pb-admin
      - edfi_ods_2021
      - edfi_ods_2022
    restart: always
    hostname: api
    container_name: ed-fi-ods-api


  db-ods-2021:
    build:
      context: ../../DB-ODS/Alpine/pgsql
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: "${POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      TPDM_ENABLED: "${TPDM_ENABLED:-true}"
      ODS_DB: "EdFi_Ods_2021"
    volumes:
      - vol-db-ods-2021:/var/lib/postgresql/data
    restart: always
    container_name: ed-fi-db-ods-2021

  edfi_ods_2021:
    image: pgbouncer/pgbouncer
    environment:
      DATABASES: "* = host = db-ods-2021 port=5432 user=${POSTGRES_USER} password=${POSTGRES_PASSWORD}"
      PGBOUNCER_LISTEN_PORT: "${PGBOUNCER_LISTEN_PORT:-6432}"
    ports:
      - "5402:${PGBOUNCER_LISTEN_PORT:-6432}"
    restart: always
    container_name: ed-fi-pb-ods-2021
    depends_on:
      - db-ods-2021

  db-ods-2022:
    build:
      context: ../../DB-ODS/Alpine/pgsql
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: "${POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      TPDM_ENABLED: "${TPDM_ENABLED:-true}"
      ODS_DB: "EdFi_Ods_2022"
    volumes:
      - vol-db-ods-2022:/var/lib/postgresql/data
    restart: always
    container_name: ed-fi-db-ods-2022

  edfi_ods_2022:
    image: pgbouncer/pgbouncer
    environment:
      DATABASES: "* = host = db-ods-2022 port=5432 user=${POSTGRES_USER} password=${POSTGRES_PASSWORD}"
      PGBOUNCER_LISTEN_PORT: "${PGBOUNCER_LISTEN_PORT:-6432}"
    ports:
      - "5403:${PGBOUNCER_LISTEN_PORT:-6432}"
    restart: always
    container_name: ed-fi-pb-ods-2022
    depends_on:
      - db-ods-2022

  adminapp:
    build:
      context: ../../Web-Ods-AdminApp/Alpine/pgsql
      dockerfile: Dockerfile
    environment:
      POSTGRES_USER: "${POSTGRES_USER}"
      POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
      POSTGRES_PORT: "${PGBOUNCER_LISTEN_PORT:-6432}"
      ODS_POSTGRES_HOST: EdFi_{0}
      ADMIN_POSTGRES_HOST: pb-admin
      API_MODE: YearSpecific
      API_EXTERNAL_URL: "https://${API_HOSTNAME:-localhost}/${ODS_VIRTUAL_NAME:-api}"
      ENCRYPTION_KEY: "${ENCRYPTION_KEY}"
      ADMINAPP_VIRTUAL_NAME: "${ADMINAPP_VIRTUAL_NAME:-adminapp}"
      API_INTERNAL_URL: ${API_INTERNAL_URL?Please consult env.example to set the Api internal url}
      ADMINAPP_HEALTHCHECK_TEST: ${ADMINAPP_HEALTHCHECK_TEST?Please consult env.example to set the Admin App healthcheck test}
      ODS_WAIT_POSTGRES_HOSTS: "EdFi_Ods_2021 EdFi_Ods_2022 "
    volumes:
      - ${LOGS_FOLDER}:/app/logs
      - adminapp-bulk-hashcache:/app/BulkUploadHashCache
      - ../../ssl:/ssl/
    depends_on:
      - pb-admin
      - api
      - edfi_ods_2021
      - edfi_ods_2022
    restart: always
    hostname: adminapp
    container_name: ed-fi-ods-adminapp
    healthcheck:
      test: $$ADMINAPP_HEALTHCHECK_TEST
      start_period: "60s"
      retries: 3

  pb-admin:
    image: pgbouncer/pgbouncer
    environment:
      DATABASES: "* = host = db-admin port=5432 user=${POSTGRES_USER} password=${POSTGRES_PASSWORD}"
      PGBOUNCER_LISTEN_PORT: "${PGBOUNCER_LISTEN_PORT:-6432}"
    ports:
      - "5401:${PGBOUNCER_LISTEN_PORT:-6432}"
    restart: always
    container_name: ed-fi-pb-admin
    depends_on:
      - db-admin

volumes:
  vol-db-admin:
    driver: local
    name: vol-db-admin
  adminapp-bulk-hashcache:
    driver: local
    name: vol-adminapp-bulk-hashcache
  vol-db-ods-2021:
    driver: local
    name: vol-db-ods-2021
  vol-db-ods-2022:
    driver: local
    name: vol-db-ods-2022
4

Since I don’t use Azure, let’s see if I can ask some questions so you can solve it for yourself eventually.

You wrote

Which product of Azure do you use then that doesn’t work? If it is not a virtual machine, then I can easily imagine that you are right when you think:

Althought it would not be a force, since “local” driver in the compose file means you want a local volume right where your containers are running. In the cloud you usually don’t want a local volume, since when your containers are moved to an other machine, you lose your data. If this is the case, Amazon should have a documentation about it. Of course when you create you containers on your VMs, you can can manage the filesystem and decide to use a local volume for example because your containers will syncronize data between multiple volumes.

So this is my guess, but as I mentioned, I don’t use Azure. If I am not right, you need someone more experienced with Azure.

5

HI Rimelek-

Apologies for the delayed response. Just returned from Holiday. After some digging, it is possible to get them to run if using an init deployment. However, it is going to be much easier to just using an Azure_File storage or direct to the Postgres we are using in a VM. I appreciate your responses and again, apologies for my delay.

Roger