Skip to content

"Pipelines for merge requests" does not work with GitLab Runner 11.8 or older versions

The feature doc => https://docs.gitlab.com/ee/ci/merge_request_pipelines/

Problem

GitLab Runner Version: 11.8.0

We are suddenly seeing all pipeline failures when being run from a Merge Request. However, we can trigger pipelines manually.

We are also seeing detached label instead of the merge request label.

We see Your runner is outdated, please upgrade your runner message on job detail pages.

Explanation: https://gitlab.com/gitlab-org/gitlab-ee/issues/11122#note_160342699

Why this happens

Tl;dr; If your GitLab environment fulfilles all of these conditions, you will encounter this problem.

  • Use Pipelines for merge requests feature. (i.e. only: [merge_requests] exists in .gitlab-ci.yml)
  • GitLab Rails version is 11.10.0 or higher (Currently, 11.10.0-rc6-ee runs on gitlab.com)
  • GitLab Runner version is 11.8.0 or lower (Currently, 11.9.0-rc2 runs on shared runners for gitlab.com)

In most cases, you're probably using specific runners with gitlab.com, and not using free-shared runners which gitlab.com provide.

Long version: In GitLab 11.10, we're going to announce the new feature "Combined Ref pipelines". This feature lets you run a pipeline on a merge ref before the actual merge happens, thus it's very userful to ensure if your pipelines for merge requests will not break target branch's pipeline.

In order to accomplish the functionality, we had to change the several core functionalities,

  • How GitLab Runner fetches and checks out the source code. Previously, GitLab Runner was able to fetch only two type of refs, one is branch refs (refs/remotes/origin/XXX) and another is tag refs (refs/tags/XXX). For accomplishing the new feature, we had to let GitLab Runner fetch merge request refs (refs/merge-requests/:iid/merge).
  • Persisting merge request ref path to pipeline.ref. Previously, it was a branch ref.

In order to run a pipeline with merge request ref, GitLab Runner has to have an ability to understand refspecs parameters sent from GitLab Rails, and old runners don't have the functionality. Thus, you're seeing Your runner is outdated, please upgrade your runner error message on job details page that you have to upgrade your runner to run pipelines correctly.

Temporary mitigation

Given there are some customers/users who use gitlab.com with their specific runners (Deployed on AWS, Digital Ocean or some cloud) and the current behavior is disturbing their workflow, we disabled ci_use_merge_request_ref feature flag on gitlab.com.

This feature flag can control if the system persists merge request ref to ci_pipelines table or not, and if it's disabled it persists a branch ref instead of a merge request ref (i.e. fallback to the old behavior).

As we've already disabled the feature flag, pipelines for merge requests can be picked by the old runners. You should not see the warning nor failed pipelines for now.

However, in order to run the new feature "Combined Ref pipelines", we need the feature flag to be enabled. Therefore, we cannot keep disabling this feature flag forever.

Upgrade Window and future plan

By the above reasons, we're going to enable this feature flag ci_use_merge_request_ref again in April 22nd 00:00 (UTC), which is the day 11.10 release blog post published.

We strongly recommend to upgrade GitLab Runners to 11.9.0 or higher until the day.

Lastly, we apologize for any inconvenience caused by this change. Please let us know if you have any questions.

Edited by Shinya Maeda