Read dependency artifact sizes from the database

What does this MR do and why?

Ci::RegisterJobService#log_build_dependencies_size (app/services/ci/register_job_service.rb:406) logs artifacts_dependencies_size and artifacts_dependencies_count into the application context for a job's dependencies. It is itself gated behind the existing ops flag ci_build_dependencies_artifacts_logger.

  • It computed the size by summing build.artifacts_file.size over the dependencies. Ci::Build#artifacts_file is the CarrierWave uploader (job_artifacts_archive&.file), and on object storage Uploader#size issues a remote object metadata request per dependency, serially, inside the runner's job request.
  • A stall in one of those requests is charged to the job request. Rack::Timeout::RequestTimeoutException descends from Exception, not StandardError, so the rescue StandardError in process_build that would drop the build with scheduler_failure never runs. By that point the build has already transitioned to running with a runner assigned, so it is left running with no runner polling it and no payload delivered, until it hits its own timeout.
  • This MR sums build.artifacts_size.to_i instead. That reads the ci_job_artifacts.size column through the same already-loaded job_artifacts_archive association, so it costs no additional queries and no network calls. .to_i handles the column being nullable at the schema level.
  • The column is safe to rely on: it is written by before_save :set_size (app/models/ci/job_artifact.rb:50) in the same INSERT that creates the artifact row, and for archive artifacts the file reaches its final location in the same transaction (after_save :store_file_in_transaction!). A visible artifact row always carries its size. The runner response in the same request already reads this size from the column, via API::Entities::Ci::JobArtifactFile serializing JobArtifactUploader#cached_size (app/uploaders/job_artifact_uploader.rb:14).

Ci::RegisterJobService#log_build_dependencies_size is already gated behind the ops flag ci_build_dependencies_artifacts_logger, which stays as the kill switch for this code path. No new flag was added: the worst case if the database value is wrong is an inaccurate log field, not a behaviour change, and the existing ops flag already removes the object storage calls entirely if it needs to be turned off.

References

  • #627240 (closed)
  • This targets the tail of request time, not the typical case. A related issue was closed as won't-do in September 2025 after a Kibana check showed the typical case is fast: #569685 (closed)

How to set up and validate locally

  1. In a rails console, enable the ops flag that turns the logger on at all:
    Feature.enable(:ci_build_dependencies_artifacts_logger)
  2. Run a pipeline where a job has a dependency with artifacts.
  3. Pick that job up with a runner (POST /api/v4/jobs/request).
  4. Check the API log for that request: json.meta.artifacts_dependencies_size should match the artifact's recorded size.
  5. bundle exec rspec spec/services/ci/register_job_service_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Hordur Freyr Yngvason

Merge request reports

Loading
Loading