Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab
GitLab
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 35,031
    • Issues 35,031
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 1,245
    • Merge Requests 1,245
  • Requirements
    • Requirements
    • List
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Metrics
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • GitLab.org
  • GitLabGitLab
  • Merge Requests
  • !17043

Merged
Opened Sep 18, 2019 by Shinya Maeda@shinya.maeda🔶Maintainer8 of 8 tasks completed8/8 tasks

Create a persistent ref per pipeline and let runners fetch it

  • Overview 78
  • Commits 1
  • Pipelines 40
  • Changes 14

What does this MR do?

This MR creates a persistent ref refs/pipelines/:iid per pipeline and let runners fetch it.

The overview of the process flow:

  • The system ensures a corresponding pipeline ref exists when build status transits to running.
  • Runners fetch code from the pipeline ref instead of source branch, refs/head or refs/merge.
  • The system cleans up the pipeline ref when pipeline finished i.e. no more active jobs.

This fix effectively resolves the following problems

  • When pipelines for merged results is enabled and source/target branch is advanced during MR pipelines are running, the pipelines fail because the target sha is not found in merge_ref_path.
  • When git-force-push is performed, it overwrites the target sha in the source ref, and result in the previous pipeline fails because source sha is not found.

Related: #14863 (closed)

Manual QA Wed Sep 25 09:30:18 UTC 2019

Case: Force-push to a branch

Scenario:

  • A developer force-pushes to the same branch three times in a short interval (< 1 sec).
  • Runner executes each pipeline and each job takes 5 seconds (sleep 5sec).
  • Git strategy is clone, thus everytime runner clones a fresh repository (No cache as fetch)

Expectation:

  • Runners successfully fetch/checkout source code from the given sha.
  • Three pipelines have successful results.
  • The persistent ref refs/pipelines/:iid have been cleaned up after pipeline finished (checked by git ls-remote)

Result: Pass

Case: Retry a job of a pipeline which ran on the non-existent sha on the associated pipeline.ref

Scenario:

  • Same as above
  • A developer retries a job of a non-latest pipeline.

Expectation:

  • The persistent ref refs/pipelines/:iid is recreated before runner works on a job.
  • Runners successfully fetch/checkout source code from the given sha.
  • The retried job have successful results.
  • The persistent ref refs/pipelines/:iid have been cleaned up after pipeline finished (checked by git ls-remote)

Result: Pass

Case: Use pipelines for merged results and target branch is advanced

Scenario:

  • "Merge pipelines will try to validate the post-merge result prior to merging" is enabled
  • A developer creates a merge request and trigger a pipeline for merged result.
  • The pipeline has three stages and each stage takes 5 seconds (Total 15 sec).
  • The target branch is advanced during the pipeline execution (git push origin target branch)

Expectation:

  • The pipeline for merged result successfully finishes. (This is the main problem to solve in #14863 (closed))

Result: Pass

Screenshots

Does this MR meet the acceptance criteria?

Conformity

  • Changelog entry
  • Documentation created/updated or follow-up review issue created
  • Code review guidelines
  • Merge request performance guidelines
  • Style guides
  • Database guides
  • Separation of EE specific content

Performance and Testing

  • Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
  • [-] Tested in all supported browsers

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited Sep 25, 2019 by Shinya Maeda
Assignee
Assign to
Reviewer
Request review from
12.4
Milestone
12.4 (Past due)
Assign milestone
Time tracking
Reference: gitlab-org/gitlab!17043
Source branch: dedicated-pipeline-ref

Revert this merge request

This will create a new commit in order to revert the existing changes.

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.