PLAT-2838_fetch-new-dois-metadata

Cristian Garciarequested to merge
PLAT-2838 into master

Added CELERY_TASK_IMPORTS so the task fetch-new-dois-metadata is recognised

TESTED locally with docker-compose as:

  • BEFORE THE CHANGES:
    $ docker compose exec metrics_api celery -A tasks.celery_app:celery_app inspect registered -\> metrics@c345580898c1: OK - empty 

    1 node online.

    $ docker compose exec metrics_api \
    celery -A tasks.celery_app:celery_app call fetch-new-dois-metadata \
    --queue metrics-api.fetch-new-dois-metadata 10f4f74a-c177-4955-b74e-46637689535f

    $ docker compose logs --tail=200 metrics_api
      ... metrics_api  | [2026-05-27 11:54:26,975: ERROR/MainProcess] Received unregistered task of type 'fetch-new-dois-metadata'.
...
metrics_api  | Traceback (most recent call last):
metrics_api  |   File "/usr/local/lib/python3.12/site-packages/celery/worker/consumer/consumer.py", line 662, in on_task_received
metrics_api  |     strategy = strategies[type_]
metrics_api  |                ~~~~~~~~~~^^^^^^^
metrics_api  | KeyError: 'fetch-new-dois-metadata'
  • AFTER THE CHANGES:
    $ docker compose exec metrics_api \
    celery -A tasks.celery_app:celery_app inspect registered -\> metrics@ec46b522242c: OK \* calculate-service-popularity-metrics \[rate_limit=1/m\] \* fetch-doi-metadata \[rate_limit=1/s\] \* fetch-new-dois-metadata \* refresh-all-popularity-metrics \* refresh-citation-cache \* refresh-event-cache \* refresh-measures-cache \* refresh-service-event-cache \* retrieve-publishers

    1 node online.

    $ docker compose exec metrics_api \
    celery -A tasks.celery_app:celery_app call fetch-new-dois-metadata \
    --queue metrics-api.fetch-new-dois-metadata fa035ffc-0787-43f1-930f-63633a5b65f9

    $ docker compose logs --tail=10 metrics_api
metrics_api  | [2026-05-27 12:00:31,615: INFO/MainProcess] metrics@ec46b522242c ready.
metrics_api  | 2026-05-27 12:00:32,616 INFO success: metrics-api entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)
metrics_api  | [2026-05-27 12:00:52,421: INFO/MainProcess] Task fetch-new-dois-metadata[fa035ffc-0787-43f1-930f-63633a5b65f9] received
metrics_api  | [2026-05-27 12:00:52,461: WARNING/ForkPoolWorker-1] 0.02 (1): 
metrics_api  |             SELECT DISTINCT event_uri FROM event
metrics_api  |                 WHERE measure_uri='https://metrics.operas-eu.org/crossref/citations/v1'
metrics_api  |             EXCEPT
metrics_api  |                 SELECT doi FROM doi_metadata;
metrics_api  |         
metrics_api  | [2026-05-27 12:00:52,462: INFO/ForkPoolWorker-1] Task fetch-new-dois-metadata[fa035ffc-0787-43f1-930f-63633a5b65f9] succeeded in 0.03999847103841603s: None

Merge request reports