WIP: Disable `move_to_cache` to make legacy trace accessible while trace artifact is being created
What does this MR do?
This problem is because of move_to_cache
option is true
in JobArtifactUploader. Users can't see the trace when the trace is in tmp/cache
or tmp/work
.
sequenceDiagram
participant Runner
participant Unicorn
participant Sidekiq
participant DB
participant Live trace path
participant tmp/work
participant tmp/cache
participant Trace artifact path
participant job page
Runner->>Unicorn: Sends chunks
loop Every minute
Unicorn->>Live trace path: Append chunks
end
activate Live trace path
Runner->>Unicorn: Sends job result
Unicorn->>Live trace path: Write full trace
Unicorn-xSidekiq: CreateTraceArtifactWorker
Note over Sidekiq,Trace artifact path: Aynsc starts
Live trace path->>tmp/work: Move file
deactivate Live trace path
activate tmp/work
Note right of tmp/work: IN THIS PERIOD, NO BODY CAN NOT ACCESS
tmp/work->>tmp/cache: Move file
deactivate tmp/work
activate tmp/cache
Note right of tmp/cache: IN THIS PERIOD, NO BODY CAN NOT ACCESS
tmp/cache->>Trace artifact path: Move file
deactivate tmp/cache
activate Trace artifact path
Sidekiq->>DB: Created trace_artifact
Note over Sidekiq,Trace artifact path: Aynsc ends
alt if trace_artifact record exists
job page->>DB: Check trace_artifact record => Not found
job page->>Live trace path: Exists?
job page->>Live trace path: Read
else if NOT trace_artifact record exists
job page->>DB: Check trace_artifact record => Found
job page->>Trace artifact path: Read
end
deactivate Trace artifact path
How this problem happens
sequenceDiagram
participant Runner
participant Unicorn
participant Sidekiq
participant DB
participant Live trace path
participant tmp/work
participant tmp/cache
participant Trace artifact path
participant job page
Runner->>Unicorn: Sends chunks
loop Every minute
Unicorn->>Live trace path: Append chunks
end
activate Live trace path
Runner->>Unicorn: Sends job result
Unicorn->>Live trace path: Write full trace
Unicorn-xSidekiq: CreateTraceArtifactWorker
Note over Sidekiq,Trace artifact path: Aynsc starts. PID: A => CreateTraceArtifactWorker. PID: B => Job page rendering
Sidekiq->>DB: Transaction starts for creating trace_artifact
job page->>DB: Check trace_artifact record => Not found (B)
job page->>Live trace path: Exists? (B)
Live trace path->>tmp/work: Move file (A)
deactivate Live trace path
activate tmp/work
job page->>Live trace path: Read (B) => Errno::ENOENT
Note right of tmp/work: IN THIS PERIOD, NO BODY CAN NOT ACCESS
tmp/work->>tmp/cache: Move file (A)
deactivate tmp/work
activate tmp/cache
Note right of tmp/cache: IN THIS PERIOD, NO BODY CAN NOT ACCESS
tmp/cache->>Trace artifact path: Move file (A)
deactivate tmp/cache
activate Trace artifact path
Sidekiq->>DB: Transaction ends for creating trace_artifact
Note over Sidekiq,Trace artifact path: Aynsc ends
deactivate Trace artifact path
-
Backport to CE
Are there points in the code the reviewer needs to double check?
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
API support added -
Tests added for this feature/bug - Review
-
Has been reviewed by UX -
Has been reviewed by Frontend -
Has been reviewed by Backend -
Has been reviewed by Database
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Squashed related commits together -
Internationalization required/considered -
If paid feature, have we considered GitLab.com plan and how it works for groups and is there a design for promoting it to users who aren't on the correct plan -
End-to-end tests pass ( package-qa
manual pipeline job)
What are the relevant issue numbers?
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/43025 Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/43022
Edited by Shinya Maeda