Should the GitLab REST API be used as a webhook receiver?

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem

In GitLab 15.7, we introduced the disable failing webhooks functionality, which disables webhooks in certain circumstances that we consider a failure. This broke a workflow for a customer, which was reported in Zendesk ticket #364978 (GitLab internal).

There's plenty of conversation detailing other users who are experiencing issues related to this change on the epic, but that's not my focus.

The issue at hand for this customer is that they were using the GitLab REST API, in particular the trigger a pipeline endpoint, as a webhook receiver. They're also using workflow rules to determine if the pipeline can run or not.

The problem is that, when workflow rules filter the pipeline, we return a 400 series response:

Hook executed successfully but returned HTTP 400 {"message":{"base":["Pipeline filtered out by workflow rules."]}}

Because the response code is a 400, this is treated as a failing webhook and, after enough failures, is disabled.

Question

As evidenced by the ticket, the REST API certainly can function as a webhook receiver, but it's unclear if we built the REST API to account for this.

In our configure your webhook receiver endpoint documentation, we say the following:

Endpoints should follow these best practices:

  • Respond quickly with a 200 or 201 status response. Avoid any significant processing of webhooks in the same request. Instead, implement a queue to handle webhooks after they are received. The timeout limit for webhooks is 10 seconds on GitLab.com.

At present, the GitLab REST API sends a 400 back when trying to trigger a pipeline that's filtered by workflow rules, which violates the above.

And so, as a result of this ticket, we (Support Engineers) pondered: Do we expect the REST API to be used as a webhook receiver?

Outcome

Depending on what we determine, we'll have a few options:

  • If the answer to the above is yes, what do we do to rectify use cases like this customer's?
  • If the answer is instead no, we should document this explicitly.
Edited by 🤖 GitLab Bot 🤖