Read dependency artifact sizes from the database
What does this MR do and why?
Ci::RegisterJobService#log_build_dependencies_size
(app/services/ci/register_job_service.rb:406) logs
artifacts_dependencies_size and artifacts_dependencies_count into
the application context for a job's dependencies. It is itself gated
behind the existing ops flag ci_build_dependencies_artifacts_logger.
- It computed the size by summing
build.artifacts_file.sizeover the dependencies.Ci::Build#artifacts_fileis the CarrierWave uploader (job_artifacts_archive&.file), and on object storageUploader#sizeissues a remote object metadata request per dependency, serially, inside the runner's job request. - A stall in one of those requests is charged to the job request.
Rack::Timeout::RequestTimeoutExceptiondescends fromException, notStandardError, so therescue StandardErrorinprocess_buildthat would drop the build withscheduler_failurenever runs. By that point the build has already transitioned torunningwith a runner assigned, so it is left running with no runner polling it and no payload delivered, until it hits its own timeout. - This MR sums
build.artifacts_size.to_iinstead. That reads theci_job_artifacts.sizecolumn through the same already-loadedjob_artifacts_archiveassociation, so it costs no additional queries and no network calls..to_ihandles the column being nullable at the schema level. - The column is safe to rely on: it is written by
before_save :set_size(app/models/ci/job_artifact.rb:50) in the sameINSERTthat creates the artifact row, and for archive artifacts the file reaches its final location in the same transaction (after_save :store_file_in_transaction!). A visible artifact row always carries its size. The runner response in the same request already reads this size from the column, viaAPI::Entities::Ci::JobArtifactFileserializingJobArtifactUploader#cached_size(app/uploaders/job_artifact_uploader.rb:14).
Ci::RegisterJobService#log_build_dependencies_size is already
gated behind the ops flag ci_build_dependencies_artifacts_logger,
which stays as the kill switch for this code path. No new flag was
added: the worst case if the database value is wrong is an
inaccurate log field, not a behaviour change, and the existing ops
flag already removes the object storage calls entirely if it needs
to be turned off.
References
- #627240 (closed)
- This targets the tail of request time, not the typical case. A related issue was closed as won't-do in September 2025 after a Kibana check showed the typical case is fast: #569685 (closed)
How to set up and validate locally
- In a rails console, enable the ops flag that turns the logger on
at all:
Feature.enable(:ci_build_dependencies_artifacts_logger) - Run a pipeline where a job has a dependency with artifacts.
- Pick that job up with a runner (
POST /api/v4/jobs/request). - Check the API log for that request:
json.meta.artifacts_dependencies_sizeshould match the artifact's recorded size. bundle exec rspec spec/services/ci/register_job_service_spec.rb
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.