PLAT-3114: Retry incomplete Crossref/DataCite prefixes in KPI script
This MR makes fetch_data_from_apis() check each prefix's fetched works against the API's own total (Crossref total-results, DataCite meta.total) and refetch any prefix that came back short or hit a failed request, up to FETCH_ATTEMPTS (3) times. In doing so, it stops a failed first Crossref page from counting as "no works" (which dropped every book under that prefix without an error), lowers MAX_WORKERS from 5 to 3, and adds a per-prefix log line plus a BOOKS breakdown: identifiers DB=… crossref=… datacite=… line to log_final_counts(), which logs an ERROR if any prefix stays incomplete.
Locally, with this change, 2026-10 gives 24,302 cumulative books (the original report had 24,278; the two prod runs on 1.2.57 gave 19,838 and 18,730) and 14,725 monthly books (originally 14,704).
- A prod
MAX_WORKERSenv var would override the new default of 3.