Fix Workhorse artifact metadata temp path falling back to os.TempDir() with direct upload
What does this MR do and why?
Fixes a bug where CI artifact uploads fail with HTTP 400 when TMPDIR is configured differently for the Rails and Workhorse processes (e.g. via gitlab_rails['env'] but not gitlab_workhorse['env'] in Omnibus).
Root cause
When direct upload to object storage is enabled, the /authorize response omits TempPath. Workhorse's artifactsUploadProcessor.tempDir is therefore empty, so generateMetadataFromZip falls back to os.TempDir() (the Workhorse process's TMPDIR) when writing the metadata.gz auxiliary file. Rails' multipart middleware (lib/gitlab/middleware/multipart.rb) then validates that path against allowed_paths, which includes Dir.tmpdir (the Rails process's TMPDIR). When these differ, UploadedFile::InvalidPathError is raised and the middleware returns HTTP 400, breaking CI artifact uploads.
Why the naive fix (always setting TempPath) is wrong
Setting TempPath unconditionally in ObjectStorage#workhorse_authorize would break Workhorse's invariant in GetOpts (workhorse/internal/upload/destination/upload_opts.go), which returns the error "API response has both TempPath and RemoteObject" when both are set. That would regress every direct-upload path (artifacts, LFS, packages, imports, dependency proxy, uploads), not just artifacts.
Fix
Generalise the solution to all direct uploads via a new LocalTempPath field:
-
Rails (
app/uploaders/object_storage.rb): inside thedirect_upload_to_object_store?branch ofworkhorse_authorize, addhash[:LocalTempPath] = Dir.tmpdir. This is returned by every direct-upload authorize endpoint (LFS, packages, imports, artifacts, etc.), not just artifacts.TempPathis still only set in the non-direct-upload branch, preserving the existingGetOptsinvariant.app/services/ci/job_artifacts/create_service.rbis net-zero versus master (no artifacts-specific field needed). -
Workhorse (
workhorse/internal/api/api.go): renameMetadataTempPathtoLocalTempPathon theResponsestruct. AddLocalTempDir()method with the precedence:TempPath→LocalTempPath→os.TempDir().TempPathwins so local-storage installs continue writing to the artifacts tmp dir exactly as they do on master. Theos.TempDir()fallback is kept for older Rails during a rolling upgrade. -
Workhorse (
workhorse/internal/upload/artifacts_uploader.go): remove themetadataTempDirfield fromartifactsUploadProcessor(struct returns to its original three fields). SettempDir: a.LocalTempDir()inArtifacts(). SimplifygenerateMetadataFromZipback tometaOpts := &destination.UploadOpts{LocalTempPath: a.tempDir}.
Spec changes
- Updated
spec/requests/api/ci/runner/jobs_artifacts_spec.rb: direct-upload context now assertsLocalTempPathequalsDir.tmpdir; local-storage context assertsLocalTempPathis absent. - Updated
spec/uploaders/object_storage_spec.rb:'uses remote storage'shared example assertsLocalTempPathequalsDir.tmpdir;'uses local storage'shared example assertsLocalTempPathis absent. - Updated all other direct-upload authorize specs that assert
not_to have_key('TempPath')to also assertLocalTempPathequalsDir.tmpdir(LFS, packages, imports, uploads, alert management, issues). - Reworked
TestMetadataTempPathPrecedence→TestLocalTempDirPrecedenceinworkhorse/internal/upload/artifacts_upload_test.goto cover the newTempPath > LocalTempPath > os.TempDir()precedence via end-to-end upload tests. Usesassert(notrequire) inside HTTP handler goroutines to satisfy testifylint. - Added
TestResponseLocalTempDirunit test inworkhorse/internal/api/api_test.gocovering all three precedence cases.
References
Screenshots or screen recordings
N/A — backend-only change.
How to set up and validate locally
- Configure object storage with direct upload enabled and set
TMPDIRdifferently for Rails vs Workhorse. - Attempt a CI artifact upload — it should succeed instead of returning HTTP 400.
- Run the relevant specs:
bin/rspec spec/requests/api/ci/runner/jobs_artifacts_spec.rb cd workhorse && go test ./internal/upload/... ./internal/api/... -v
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.