Skip to content

feat: add referrers data to internal List Tags API

What does this MR do?

Extends the List Tags API to include referrer info for each tag - i.e. each manifest that has a subject reference to the tag.

  • Adds a referrers=true query option to the List Tags endpoint. The value must be true for referrers to be included.
  • For each tag that would be output, the referrer info (if present) is appended in this format:
    ...
    "referrers": [
      {
        "artifactType": "application/vnd.dev.cosign.artifact.sig.v1+json",
        "digest": "sha256:eb110b254f038cd0d464b20de5070099a6f675b9d8eb7e5035d929540a041ab1"
      },
      ...
    ]
    ...

Implementation Notes

  • models.TagDetail was extended with the Referrers field.
  • The results from tagsDetailPaginatedQuery are appended to by the new appendTagsDetailReferrers method. This means referrer data should be queried on the already-paginated tag results.
  • The new query takes a list of manifest digests mapped from the given tags, and returns a list of manifests which are referrers for those tags. Those referrers are then mapped back to their target manifests (specified by subject_id) and appended to the result.
  • COALESCE(at.media_type,cmt.media_type) is used as the final artifactType per OCI spec on referrer lists. If artifactType is not present on the referrer, we fall back to the configuration media type.
Query
SELECT
	encode(m.digest, 'hex') AS digest,
	COALESCE(at.media_type,cmt.media_type) AS artifact_type,
	encode(ms.digest, 'hex') AS subject_digest
FROM manifests AS m
JOIN manifests AS ms ON m.top_level_namespace_id = ms.top_level_namespace_id
	AND m.subject_id = ms.id
	LEFT JOIN media_types AS at ON at.id = m.artifact_media_type_id
	LEFT JOIN media_types AS cmt ON cmt.id = m.configuration_media_type_id
WHERE
	m.top_level_namespace_id = $1
	AND m.repository_id = $2
	AND m.subject_id IN (
		SELECT id
		FROM manifests
		WHERE
			top_level_namespace_id = $1
			AND repository_id = $2
			AND digest IN (
				SELECT decode(n, 'hex')
				FROM unnest(ARRAY[%s]) AS n

Query plan: https://console.postgres.ai/gitlab/gitlab-production-registry/sessions/24409/commands/77966

Author checklist

  • Feature flags
    • Added feature flag:
    • This feature does not require a feature flag
  • I added unit tests or they are not required
  • I added documentation (or it's not required)
  • I followed code review guidelines
  • I followed Go Style guidelines
  • For database changes including schema migrations:
    • Manually run up and down migrations in a postgres.ai production database clone and post a screenshot of the result here.
    • If adding new queries, extract a query plan from postgres.ai and post the link here. If changing existing queries, also extract a query plan for the current version for comparison.
    • Do not include code that depends on the schema migrations in the same commit. Split the MR into two or more.
  • Ensured this change is safe to deploy to individual stages in the same environment (cny -> prod). State-related changes can be troublesome due to having parts of the fleet processing (possibly related) requests in different ways.

Reviewer checklist

  • Ensure the commit and MR tittle are still accurate.
  • If the change contains a breaking change, apply the breaking change label.
  • If the change is considered high risk, apply the label high-risk-change
  • Identify if the change can be rolled back safely. (note: all other reasons for not being able to rollback will be sufficiently captured by major version changes).

If the MR introduces database schema migrations:

  • Ensure the commit and MR tittle start with fix:, feat:, or perf: so that the change appears on the Changelog
If the changes cannot be rolled back follow these steps:
  • If not, apply the label cannot-rollback.
  • Add a section to the MR description that includes the following details:
    • The reasoning behind why a release containing the presented MR can not be rolled back (e.g. schema migrations or changes to the FS structure)
    • Detailed steps to revert/disable a feature introduced by the same change where a migration cannot be rolled back. (note: ideally MRs containing schema migrations should not contain feature changes.)
    • Ensure this MR does not add code that depends on these changes that cannot be rolled back.

Related to #1009 (closed)

Edited by Aaron Huntsman

Merge request reports