Skip to content

Use lazy evaluation when triggering project hooks

Luke Duncalfe requested to merge 326528-lazy-evaluate-webhooks into master

What does this MR do and why?

This MR adds a new class to enforce best-practice performance when triggering project hooks, and uses it behind a feature flag for issue hooks.

Problem

We have many places in our codebase where we call webhooks and integration hooks, and build the data payload beforehand. This can be expensive if we need to fetch associations, and redundant if we don't actually have any hooks/services to call.

Improvement

The new class takes a payload block, and checks that hooks need to be executed using the existing Project#has_active_hooks? and Project#has_integration_hooks? checks before calling the block to load the payload. This happens in request to ensure the payload state is as current as can be.

A worker continues to load the hooks out of the request (doing what Project#execute_hooks and Project#execute_integrations currently do).

The issue hooks were already being triggered reasonably performantly. We're rolling this change out initially to the issue hooks because they're triggered less often than most, and lets us roll out cautiously. Other hooks will benefit more from these changes &8909 (comment 1126063325) and the next step will be to roll them out to other hook types. We'll eventually remove the ability to trigger project hooks outside of the new class #376248.

Tests

The bulk of the MR changes are to specs. The specs now include tests that assert the relevant contents of the webhook payloads, as well as that hooks are triggered. This gives us some assurance that the change in this MR builds equivalent payloads, but also forms a basis of test best-practice for webhooks that we'll document later #377490.

&8909 (comment 1123076852)

#326528

How to set up and validate locally

  1. Enable the flag: 
    Feature.enable(:web_hooks_optimize_issue_hook)
  2. Visit https://webhook.site to generate a new webhook receiving endpoint to test with.
  3. Configure a webhook for a project:
    1. Go Settings > Webhooks.
    2. For URL paste in the URL generated for you by https://webhook.site.
    3. For Trigger, check Issues events and Confidential issues events.
    4. Click Add webhooks.
  4. Observe the payloads when you mutate an issue (create, update, close, reopen, move).
  5. Do the same with a confidential issue.
  6. Disable the flag.
  7. Observe the payloads when you mutate an issue (create, update, close, reopen, move), they should be the same.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #326528

Edited by Luke Duncalfe

Merge request reports