Use p_ci_job_artifacts table for exposed artifacts data

What does this MR do and why?

Context

options => artifacts:expose_as is needed long term for the MR widgets exposing artifacts feature. This should be considered intrinsic data because it's used in MR widgets to display artifacts. Currently this data is stored and read from p_ci_builds_metadata.config_options, but this is a only a short term destination.

For long term storage, we decided to store this intrinsic data in p_ci_job_artifacts. And in !197728 (merged), we added the new columns exposed_as and exposed_paths.

This MR

With the new columns in place, this MR now implements the next step of #549079 (closed), which is to write/read from p_ci_job_artifacts for exposed artifacts.

Details:

• Stores the artifacts:expose_as and artifacts:paths values into p_ci_job_artifacts upon persisting a new metadata artifact.
• When finding exposed artifacts, it first attempts to read from p_ci_job_artifacts; if the data is not present there, then it defaults to reading from p_ci_builds_metadata as it does currently.
  • This enables us to support both migrated and non-migrated exposed artifacts data. The data migration of existing jobs will be performed as part of Create BBM to migrate intrinsic artifacts:expos... (#555430 - closed).
  • Note that we only read from these new columns for the MR UI widget feature, i.e only after the job and artifacts have persisted. During job processing, we still obtain the values from job.options.

These changes are made under a Feature Flag: ci_use_job_artifacts_table_for_exposed_artifacts

Note: This feature flag will not be rolled out until after the exposed_as column's text constraint has been validated on GitLab.com (which should be happening asynchronously this weekend, July 26-27, 2025).

References

• Resolves 2nd step of: Store `artifacts:expose_as` into a different table (#549079 - closed)
• FF Roll out issue: [FF] `ci_use_job_artifacts_table_for_exposed_ar... (#557295 - closed)

Database queries

Here are the new preloading queries introduced by this MR. Note that the existing batching/preloading queries were reviewed back in !195693 (merged).

Query 1: Project preload

Raw SQL

SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_last_update_at", "projects"."mirror_last_successful_update_at", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."pull_mirror_branch_prefix", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id", "projects"."hidden", "projects"."organization_id"
FROM "projects"
WHERE "projects"."id" = 278964;

Query plan: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/41826/commands/128332

Query 2: job_artifacts_metadata preload

Raw SQL

SELECT
    "p_ci_job_artifacts"."job_id",
    "p_ci_job_artifacts"."partition_id",
    "p_ci_job_artifacts"."exposed_as"
FROM
    "p_ci_job_artifacts"
WHERE
    "p_ci_job_artifacts"."partition_id" = 105
    AND "p_ci_job_artifacts"."file_type" = 2
    AND "p_ci_job_artifacts"."job_id" IN (10806402556, 10806399061, 10806403700, 10806398249);

Query plan: https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/41828/commands/128336

How to set up and validate locally

These steps will lead you through creating an exposed artifact.

Prerequisite: Ensure you have a GitLab Runner set up with your gdk.

1. Enable the feature flag in Rails console: Feature.enable(:ci_use_job_artifacts_table_for_exposed_artifacts)

2. Create a new blank project with the following pipeline config:

.gitlab-ci.yml

generate-exposed-artifact:
  script:
    - echo "My test artifact" > my_artifact.txt
  artifacts:
    paths:
      - my_artifact.txt
    expose_as: 'My Test Artifact'

control-job:
  script: echo

3. Commit (anything) to a new branch and open a merge request. In that MR, wait for the pipeline to complete. Then observe that the artifact is correctly exposed in the UI:

4. Now delete the metadata records for the jobs in the pipeline from the previous step. This is to ensure that the data is read correctly from p_ci_job_artifacts even when the job's processing data is removed.

pipeline = Ci::Pipeline.find(<YOUR_PIPELINE_ID>)
pipeline.builds.each { |b| b.metadata.delete }

5. Refresh the merge request page (from Step 3) and you should still see the exposed artifacts in the UI as before.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #549079 (closed)