Fix Workhorse artifact metadata temp path falling back to os.TempDir() with direct upload

What does this MR do and why?

Fixes a bug where CI artifact uploads fail with HTTP 400 when TMPDIR is configured differently for the Rails and Workhorse processes (e.g. via gitlab_rails['env'] but not gitlab_workhorse['env'] in Omnibus).

Root cause

When direct upload to object storage is enabled, the /authorize response omits TempPath. Workhorse's artifactsUploadProcessor.tempDir is therefore empty, so generateMetadataFromZip falls back to os.TempDir() (the Workhorse process's TMPDIR) when writing the metadata.gz auxiliary file. Rails' multipart middleware (lib/gitlab/middleware/multipart.rb) then validates that path against allowed_paths, which includes Dir.tmpdir (the Rails process's TMPDIR). When these differ, UploadedFile::InvalidPathError is raised and the middleware returns HTTP 400, breaking CI artifact uploads.

Why the naive fix (always setting TempPath) is wrong

Setting TempPath unconditionally in ObjectStorage#workhorse_authorize would break Workhorse's invariant in GetOpts (workhorse/internal/upload/destination/upload_opts.go), which returns the error "API response has both TempPath and RemoteObject" when both are set. That would regress every direct-upload path (artifacts, LFS, packages, imports, dependency proxy, uploads), not just artifacts.

Fix

Generalise the solution to all direct uploads via a new LocalTempPath field:

  • Rails (app/uploaders/object_storage.rb): inside the direct_upload_to_object_store? branch of workhorse_authorize, add hash[:LocalTempPath] = Dir.tmpdir. This is returned by every direct-upload authorize endpoint (LFS, packages, imports, artifacts, etc.), not just artifacts. TempPath is still only set in the non-direct-upload branch, preserving the existing GetOpts invariant. app/services/ci/job_artifacts/create_service.rb is net-zero versus master (no artifacts-specific field needed).

  • Workhorse (workhorse/internal/api/api.go): rename MetadataTempPath to LocalTempPath on the Response struct. Add LocalTempDir() method with the precedence: TempPath → LocalTempPath → os.TempDir(). TempPath wins so local-storage installs continue writing to the artifacts tmp dir exactly as they do on master. The os.TempDir() fallback is kept for older Rails during a rolling upgrade.

  • Workhorse (workhorse/internal/upload/artifacts_uploader.go): remove the metadataTempDir field from artifactsUploadProcessor (struct returns to its original three fields). Set tempDir: a.LocalTempDir() in Artifacts(). Simplify generateMetadataFromZip back to metaOpts := &destination.UploadOpts{LocalTempPath: a.tempDir}.

Spec changes

  • Updated spec/requests/api/ci/runner/jobs_artifacts_spec.rb: direct-upload context now asserts LocalTempPath equals Dir.tmpdir; local-storage context asserts LocalTempPath is absent.
  • Updated spec/uploaders/object_storage_spec.rb: 'uses remote storage' shared example asserts LocalTempPath equals Dir.tmpdir; 'uses local storage' shared example asserts LocalTempPath is absent.
  • Updated all other direct-upload authorize specs that assert not_to have_key('TempPath') to also assert LocalTempPath equals Dir.tmpdir (LFS, packages, imports, uploads, alert management, issues).
  • Reworked TestMetadataTempPathPrecedence → TestLocalTempDirPrecedence in workhorse/internal/upload/artifacts_upload_test.go to cover the new TempPath > LocalTempPath > os.TempDir() precedence via end-to-end upload tests. Uses assert (not require) inside HTTP handler goroutines to satisfy testifylint.
  • Added TestResponseLocalTempDir unit test in workhorse/internal/api/api_test.go covering all three precedence cases.

References

Screenshots or screen recordings

N/A — backend-only change.

How to set up and validate locally

  1. Configure object storage with direct upload enabled and set TMPDIR differently for Rails vs Workhorse.
  2. Attempt a CI artifact upload — it should succeed instead of returning HTTP 400.
  3. Run the relevant specs:
    bin/rspec spec/requests/api/ci/runner/jobs_artifacts_spec.rb
    cd workhorse && go test ./internal/upload/... ./internal/api/... -v

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Duo Developer

Merge request reports

Loading
Loading