Skip to content

Processors - Check for an existing comment last

Context

Some processors take up to 90 seconds to just run their applicable? or comment_cleanup_applicable? method. This happens when:

  1. The issue has A LOT of comments (e.g. this one)
  2. We're using UniqueComment to see whether we already posted a comment from that processor (we have to go through ALL the comments in the issue)

Closes Three processors causing 4.5-minute webhook pro... (#1668 - closed).

What does this MR do and why?

  1. We do API calls at the very end for applicable? and comment_cleanup_applicable? methods, which are called for every processor, for every webhooks.
    • This reduces the 4 webhooks I used for testing from 90s for each of the three problematic processors to ~1-2s for each 🚀
  2. We filter certain kinds of webhooks that were triggered by automation

Other things I tried

I tried to search via the https://docs.gitlab.com/api/search/ API, but:

  1. It would return ALL the issues matching the hidden comment we add for a processor (potentially thousands of issues). We cannot search AND filter for a specific issue at the same time it seems.
  2. I thought of changing the hidden UniqueComment comment from processor_name to also include the issue/MR ID. It is not worth pursuing at the time, as I expect this MR to fix 90+% of the long-webhook-processing cases.

Expected impact & dry-runs

  1. Faster webhook processing overall when issues have a lot of comments (applicable issues with a lot of comments would still be slow)
  2. A lot less API calls

Action items

Edited by David Dieulivol

Merge request reports

Loading