[4.5.7] Celery ImportError, backend_cleanup is not defined, DEBUG/TESTING
I thought about not posting this issue, since while typing out the description, I think I found out the root cause of it, but not the solution though. I am still creating it, because this may be useful to other people that run Mayan, and maybe the maintainers of Mayan as well, as this technically is still an error, even though it can only be observed while running Mayan with DEBUG mode.
I have the following error after migrating to 4.5.7 from 4.4.6, full container log celery_error.txt (log is from separate, clean install), excerpt of error from it:
2023-11-28 11:29:59 [2023-11-28 09:29:59,956: ERROR/MainProcess] Cannot add entry 'celery.backend_cleanup' to database schedule: ImportError('Module "celery" does not define a "backend_cleanup" attribute/class'). Contents: {'task': 'celery.backend_cleanup', 'schedule': <crontab: 0 4 * * * (m/h/dM/MY/d)>, 'options': {'expire_seconds': 43200}}
2023-11-28 11:29:59 Traceback (most recent call last):
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/models/query.py", line 581, in get_or_create
2023-11-28 11:29:59 return self.get(**kwargs), False
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/models/query.py", line 435, in get
2023-11-28 11:29:59 raise self.model.DoesNotExist(
2023-11-28 11:29:59 django_celery_beat.models.PeriodicTask.DoesNotExist: PeriodicTask matching query does not exist.
2023-11-28 11:29:59
2023-11-28 11:29:59 During handling of the above exception, another exception occurred:
2023-11-28 11:29:59
2023-11-28 11:29:59 Traceback (most recent call last):
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/utils/module_loading.py", line 20, in import_string
2023-11-28 11:29:59 return getattr(module, class_name)
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/celery/local.py", line 469, in __getattr__
2023-11-28 11:29:59 return ModuleType.__getattribute__(self, name)
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 AttributeError: module 'celery' has no attribute 'backend_cleanup'
2023-11-28 11:29:59
2023-11-28 11:29:59 The above exception was the direct cause of the following exception:
2023-11-28 11:29:59
2023-11-28 11:29:59 Traceback (most recent call last):
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django_celery_beat/schedulers.py", line 317, in update_from_dict
2023-11-28 11:29:59 entry = self.Entry.from_entry(name,
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django_celery_beat/schedulers.py", line 176, in from_entry
2023-11-28 11:29:59 return cls(PeriodicTask._default_manager.update_or_create(
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method
2023-11-28 11:29:59 return getattr(self.get_queryset(), name)(*args, **kwargs)
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django_celery_beat/managers.py", line 10, in update_or_create
2023-11-28 11:29:59 obj, created = self.get_or_create(defaults=defaults, **kwargs)
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/models/query.py", line 588, in get_or_create
2023-11-28 11:29:59 return self.create(**params), True
2023-11-28 11:29:59 ^^^^^^^^^^^^^^^^^^^^^
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/db/models/query.py", line 453, in create
2023-11-28 11:29:59 obj.save(force_insert=True, using=self.db)
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/task_manager/methods.py", line 6, in method_save
2023-11-28 11:29:59 import_string(dotted_path=self.task)
2023-11-28 11:29:59 File "/opt/mayan-edms/lib/python3.11/site-packages/django/utils/module_loading.py", line 22, in import_string
2023-11-28 11:29:59 raise ImportError('Module "%s" does not define a "%s" attribute/class' % (
2023-11-28 11:29:59 ImportError: Module "celery" does not define a "backend_cleanup" attribute/class
Tried repeating the error on versions:
- v4.5.7 - error
- v4.4.9 - error
- v4.4.6 - no error
The error does not prevent Mayan from running, you can still open the site, but it is concerning as I assume "backend_cleanup" might be an essential background job, even though the ImportError
only appears with debugs turned on.
I also found this, 'File "/opt/mayan-edms/lib/python3.11/site-packages/mayan/apps/task_manager/methods.py", line 6, in method_save' is the function factory_method_periodic_task_save
, which is only called here https://gitlab.com/mayan-edms/mayan-edms/-/blob/v4.5.7/mayan/apps/task_manager/apps.py?ref_type=tags#L106, where it is also only called if the following expression is true settings.DEBUG or settings.TESTING
, the commit I found that introduced this is 716c7ded.
I think this error is probably not an issue at all for most users, since it looks like it can only occur when running Mayan with DEBUG/TESTING turned on (which I am on the test environment that I have). I assume this should not affect production environments at all. But, still, the error is there in test environments, and you may or may not want to fix the issue. I assume that "periodic task import checking" is not an essential job at all then.
I also found this comment in a closed issue: #1156 (comment 1589659794), it is unrelated with the original #1156 (closed), and there is no followup on what is the result of the investigation.
how to repeat issue
The error can only be seen if you turn on debugging with DEBUG=True
in the settings files, minimum settings file with which the error can be repeated:
# only_debug.py
from mayan.settings.production import * # NOQA
DEBUG = True
Repeated the error with clean install using docker, adding commands I used to run images, you must use Redis, without it, the error does not appear, see docker commands:
docker run --hostname=c39ed615cdd7 --mac-address=02:42:ac:11:00:03 --env=REDIS_PASSWORD=mayanredispassword --env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --env=REDIS_VERSION=7.0.12 --env=REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.0.12.tar.gz --env=REDIS_DOWNLOAD_SHA=9dd83d5b278bb2bf0e39bfeb75c3e8170024edbaf11ba13b7037b2945cf48ab7 --volume=/data --workdir=/data -p 6379:6379 --runtime=runc -d redis:7.0.12-alpine
docker run --hostname=f82944669ccb --user=root --mac-address=02:42:ac:11:00:02 --env=MAYAN_STATIC_ROOT=/opt/mayan-edms/static --env=MAYAN_SETTINGS_MODULE=mayan.settings.only_debug --env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin --env=LC_ALL=C.UTF-8 --env=PYTHONUNBUFFERED=1 --env=PROJECT_INSTALL_DIR=/opt/mayan-edms/ --env=MAYAN_MEDIA_ROOT=/var/lib/mayan --volume=C:\only_debug.py:/opt/mayan-edms/lib/python3.11/site-packages/mayan/settings/only_debug.py --volume=/var/lib/mayan -p 8000:8000 --restart=no --runtime=runc -d mayanedms/mayanedms:v4.5.7
I am using Windows to repeat the issue with a clean install, so you may or may not have to use a different docker run command. I assume you have your own scripts to run Mayan, important bit here is to specify the custom settings file with DEBUG=True
, and to use Redis with Mayan.