perf(readers): MetadataSet read re-resolves MetadataStructure on every call (regression after #72 fix)

Summary

In 0.27.1, Registry.get(<MetadataSet URN>) triggers an eager full-read of the referenced MetadataStructure via _resolve_refs → _resolve_urns → _read_metadata_structure. When a caller iterates many MetadataSets that share the same MSD (e.g. listing all 293 entries of a Metadataflow), the same MSD is re-read N times — same shape as #72 (closed) / fixed-then, but on a different reader path that the memoisation in 0.27.1 didn't cover.

End-to-end impact on cpt-sdmx-api /v1/metadata/metadataflow/STC_CENSUS/MDF_CP_DICTIONARY/1.0?limit=N&detail=full:

N	0.27.0	0.27.1	Regression
5	100 ms	290 ms	2.9×
50	540 ms	2.5 s	5×
293 (all)	2.9 s	15.1 s	5×

Repro

import time, cProfile, pstats, sdmxlib as sl

reg = ...  # Registry open against a bundle with N MetadataSets sharing one MSD
ms_urn = sl.MetadataSet.urn(agency="STC_CENSUS", id="MS_DICT_DWELLING_LOGEMENTS001", version="1.0")
reg.get(ms_urn)  # prime

# Single read
t = time.perf_counter()
reg.get(ms_urn)
print(f"single MetadataSet read: {(time.perf_counter()-t)*1000:.1f} ms")  # ~70 ms — 45 ms of which is _read_metadata_structure

# Re-iterate over many sets that share the same MSD
ms_urns = [...]  # e.g. all 293 MS_DICT_* sets from MDF_CP_DICTIONARY
t = time.perf_counter()
for u in ms_urns:
    reg.get(u)
print(f"{len(ms_urns)} sets: {(time.perf_counter()-t):.2f} s")  # ~13 s instead of <1 s

Profile evidence

cProfile of 10× reg.get(<MetadataSet URN>) on sdmxlib 0.27.1:

       ncalls  cumtime  filename:lineno(function)
           10    0.711   _registry.py:242(get)
           30    0.703   readers.py:1381(get_artefact)
          200    0.686   _duckdb.execute
           10    0.573   _registry.py:279(_resolve_refs)
           20    0.565   _registry.py:372(_resolve_urns)
           10    0.456   readers.py:1058(_read_metadata_structure)   ← THE CULPRIT
           10    0.399   readers.py:157(_read_annotations_subtree)
           10    0.129   readers.py:1252(_read_metadata_set)

_read_metadata_structure runs on every MetadataSet read despite the MSD URN being identical across hundreds of consecutive reg.get() calls. The MSD itself materialises in ~45 ms (with the _read_annotations_subtree bulk-load fix from #73 (closed) working as designed — that part is fine).

Root cause

Same shape as #72 (closed) but the resolver fan-out is now along MetadataSet → metadataflow_urn → metadata_structure_urn → MetadataStructure. The fix that landed for #72 (closed) memoises ConceptScheme/Codelist/Hierarchy reads within a _resolve_refs walk, but MetadataStructure (and the intermediate Metadataflow) Refs are not in the memoised set, so they pay full re-resolution per outer call.

Suggested fixes

Any of these closes the regression:

Extend the _resolve_refs memoisation map to include MetadataStructure and Metadataflow URNs alongside the artefact classes already covered for #72 (closed).
Skip MSD resolution during MetadataSet reads. A MetadataSet carries the MSD URN as a Ref; the consumer can resolve it explicitly if they need the structure. Mirrors the Ref[DataStructure].unresolved(...) pattern already used by _read_dataflow.
Cache reader outputs at the Registry level by URN (LRU). Broader but addresses every future variant of this bug.

(1) is the minimal/local fix. (2) is the design-aligned fix that matches how Dataflow → DSD refs are handled today (kept unresolved). Both work.

Expected impact

After fix: dict listing of 293 entries drops from 15.1 s → ~2 s (linear in N × _read_metadata_set cost, with a single MSD read amortised across all sets).

Environment

sdmxlib 0.27.1
DuckDB 1.5.x
Python 3.13 / 3.14
Bundle: cenprof-2021 (MDF_CP_DICTIONARY → 293 MS_DICT_* sets all referencing MSD_CP_DICTIONARY)

#72 (closed) — _resolve_refs fan-out (fixed in 0.27.1 for ConceptScheme/Codelist/Hierarchy, but the memoisation didn't cover MSD/Metadataflow)
#73 (closed) — _read_annotations N+1 (fixed in 0.27.1 via _read_annotations_subtree; visible in this profile, working correctly)