Docker Community Forums

Share and learn in the Docker community.

Docker + celery starts with an error but then works fine

an error occurs when starting celery:

dmndmn-celery_worker-1  | Usage: celery [OPTIONS] COMMAND [ARGS]...
dmndmn-celery_worker-1  | Try 'celery --help' for help.
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  | Error: Invalid value for '-A' / '--app': 
dmndmn-celery_worker-1  | Unable to load celery application.
dmndmn-celery_worker-1  | While trying to load the module dmndmn.celery_run.celery the following error occurred:
dmndmn-celery_worker-1  | Traceback (most recent call last):
dmndmn-celery_worker-1  |   File "/usr/local/lib/python3.8/dist-packages/celery/bin/celery.py", line 53, in convert
dmndmn-celery_worker-1  |     return find_app(value)
dmndmn-celery_worker-1  |   File "/usr/local/lib/python3.8/dist-packages/celery/app/utils.py", line 384, in find_app
dmndmn-celery_worker-1  |     sym = symbol_by_name(app, imp=imp)
dmndmn-celery_worker-1  |   File "/usr/local/lib/python3.8/dist-packages/kombu/utils/imports.py", line 56, in symbol_by_name
dmndmn-celery_worker-1  |     module = imp(module_name, package=package, **kwargs)
dmndmn-celery_worker-1  |   File "/usr/local/lib/python3.8/dist-packages/celery/utils/imports.py", line 100, in import_from_cwd
dmndmn-celery_worker-1  |     return imp(module, package=package)
dmndmn-celery_worker-1  |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
dmndmn-celery_worker-1  |     return _bootstrap._gcd_import(name[level:], package, level)
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
dmndmn-celery_worker-1  |   File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
dmndmn-celery_worker-1  | ModuleNotFoundError: No module named 'dmndmn'

but after that celery restarts fine and works:

dmndmn-celery_worker-1  |  -------------- celery@4a2724c340e4 v5.2.0 (dawn-chorus)
dmndmn-celery_worker-1  | --- ***** ----- 
dmndmn-celery_worker-1  | -- ******* ---- Linux-5.13.19-2-MANJARO-x86_64-with-glibc2.29 2021-11-08 19:42:40
dmndmn-celery_worker-1  | - *** --- * --- 
dmndmn-celery_worker-1  | - ** ---------- [config]
dmndmn-celery_worker-1  | - ** ---------- .> app:         dmndmn:0x7f7e3ce3c250
dmndmn-celery_worker-1  | - ** ---------- .> transport:   redis://redis:6379/0
dmndmn-celery_worker-1  | - ** ---------- .> results:     redis://redis:6379/0
dmndmn-celery_worker-1  | - *** --- * --- .> concurrency: 5 (prefork)
dmndmn-celery_worker-1  | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
dmndmn-celery_worker-1  | --- ***** ----- 
dmndmn-celery_worker-1  |  -------------- [queues]
dmndmn-celery_worker-1  |                 .> celery           exchange=celery(direct) key=celery
dmndmn-celery_worker-1  |                 
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  | [tasks]
dmndmn-celery_worker-1  |   . dmndmn.tasks.start_scan_task
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  | [2021-11-08 19:42:40,480: WARNING/MainProcess] /usr/local/lib/python3.8/dist-packages/celery/app/utils.py:204: CDeprecationWarning: 
dmndmn-celery_worker-1  |     The 'CELERY_RESULT_BACKEND' setting is deprecated and scheduled for removal in
dmndmn-celery_worker-1  |     version 6.0.0. Use the result_backend instead
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  |   deprecated.warn(description=f'The {setting!r} setting',
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  | [2021-11-08 19:42:40,480: WARNING/MainProcess] /usr/local/lib/python3.8/dist-packages/celery/app/utils.py:204: CDeprecationWarning: 
dmndmn-celery_worker-1  |     The 'CELERY_IMPORTS' setting is deprecated and scheduled for removal in
dmndmn-celery_worker-1  |     version 6.0.0. Use the imports instead
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  |   deprecated.warn(description=f'The {setting!r} setting',
dmndmn-celery_worker-1  | 
dmndmn-celery_worker-1  | [2021-11-08 19:42:40,480: WARNING/MainProcess] Please run `celery upgrade settings path/to/settings.py` to avoid these warnings and to allow a smoother upgrade to Celery 6.0.
dmndmn-celery_worker-1  | [2021-11-08 19:42:40,616: INFO/MainProcess] Connected to redis://redis:6379/0
dmndmn-celery_worker-1  | [2021-11-08 19:42:40,621: INFO/MainProcess] mingle: searching for neighbors
dmndmn-celery_worker-1  | [2021-11-08 19:42:41,634: INFO/MainProcess] mingle: all alone
dmndmn-celery_worker-1  | [2021-11-08 19:42:41,665: INFO/MainProcess] celery@4a2724c340e4 ready.
dmndmn-celery_worker-1  | [2021-11-08 19:42:45,608: INFO/MainProcess] Events of group {task} enabled by remote.
dmndmn-celery_worker-1  | [2021-11-08 19:44:08,716: INFO/MainProcess] Task dmndmn.tasks.start_scan_task[7248dc6b-88f3-472c-b07d-ccff09ae0d79] received
dmndmn-celery_worker-1  | [2021-11-08 19:44:08,933: WARNING/ForkPoolWorker-4] 
dmndmn-celery_worker-1  | OneForAll is a powerful subdomain integration tool

command:
celery -A dmndmn.celery_run.celery worker --loglevel=info --concurrency=5

celery_run.py:

from celery import Celery
from dotenv import dotenv_values
from . import create_app


def make_celery(app):
    # create context tasks in celery
    celery = Celery(app.import_name, broker=app.config["ENV_FILE"]["CELERY_BROKER_URL"])
    celery.conf.update(app.config["ENV_FILE"])
    """uncomment for non-pereodical call tasks"""
    TaskBase = celery.Task

    class ContextTask(TaskBase):
        abstract = True

        def __call__(self, *args, **kwargs):
            with app.app_context():
                return TaskBase.__call__(self, *args, **kwargs)

    celery.Task = ContextTask
    return celery


app = create_app()
celery = make_celery(app)

structure dirs:

- dmndmn:
    - celery_run.py
    - __init__.py (main application and create_app flask)
- docker-compose.yml
- Dockerfile
- celery_entrypoint.sh

why i’m getting. is this error persistent?