Swap to UNLINK for Redis set cache

What does this MR do?

This uses UNLINK in Redis 4 to delete sets. This unlinks the key from the value but actually deletes the value asynchronously. With large sets this should give us a reasonable performance boost.

This MR also implements multi-key deletes for RepositorySetCache methods in the Repository model, meaning only one UNLINK call will be issued for all sets being expired.

Implements :repository_set_cache_unlink feature flag, defaulting to on.

Screenshots

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

changed milestone to %12.9

added bugperformance devopscreate groupsource code labels

changed the description

@igorwwwwwwwwwwwwwwwwwwww here's a first usage of UNLINK; it's pretty easy to implement on the set cache so figure it'd be a good place to start.

I'll add a feature flag around this before I send it off for review.

Reviewer roulette

Changes that require review have been detected! A merge request is normally reviewed by both a reviewer and a maintainer in its primary category (e.g. frontend or backend), and by a maintainer in all other categories.

To spread load more evenly across eligible reviewers, Danger has randomly picked a candidate for each review slot. Feel free to override this selection if you think someone else would be better-suited, or the chosen person is unavailable.

To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines.

Once you've decided who will review this merge request, mention them as you normally would! Danger does not (yet?) automatically notify them for you.

Category	Reviewer	Maintainer
backend	Gabriel Mazetto (@brodock)	Mayra Cabrera (@mayra-cabrera)

Generated by Danger

added typefeature label

added 1 commit

• c1677355 - Swap to UNLINK for Redis set cache

Compare with previous version

marked the checklist item Changelog entry as completed

added 1 commit

• bae909b4 - Swap to UNLINK for Redis set cache

Compare with previous version

Passing this to @andrewn for review as one of the few people who have worked on the RepositorySetCache stuff before who is also not on holiday/at capacity

assigned to @andrewn and unassigned @robotmay

unmarked as a Work In Progress

changed the description

We discovered in https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/9420#note_304486361 that there is a lazyfree-lazy-server-del option that should allow us to configure this behaviour for DEL on the server side.

@robotmay this is great. Lets get it turned on and trying this out as soon as possible. It should hopefully make a big difference to the volume of slowlog entries being generated.

approved this merge request

assigned to @robotmay and unassigned @andrewn

assigned to @stanhu and unassigned @robotmay

Passing to @stanhu for maintainer review

assigned to @robotmay and unassigned @stanhu

added 1 commit

• 972b5e36 - Gracefully fall back to DEL

Compare with previous version

added 257 commits

• 972b5e36...c925fbfd - 255 commits from branch master
• eb088a4d - Swap to UNLINK for Redis set cache
• f6019c62 - Gracefully fall back to DEL

Compare with previous version

added 1 commit

• 69a4ae85 - No need to glob keys again

Compare with previous version

added 185 commits

• 69a4ae85...dcb0e69f - 182 commits from branch master
• d3eb48dd - Swap to UNLINK for Redis set cache
• 0e4116e6 - Gracefully fall back to DEL
• cc4d9af9 - No need to glob keys again

Compare with previous version

assigned to @stanhu and unassigned @robotmay

Back to you @stanhu, as I've sorted out the version issue and rebased again

resolved all threads

added 49 commits

• cc4d9af9...960eff07 - 46 commits from branch master
• 568905a0 - Swap to UNLINK for Redis set cache
• 474ed146 - Gracefully fall back to DEL
• b7cff7f5 - No need to glob keys again

Compare with previous version

approved this merge request

enabled an automatic merge when the pipeline for ba20751d succeeds

resolved all threads

merged

mentioned in commit 5cd3a67a

added workflowstaging label

added workflowcanary label and removed workflowstaging label

added workflowproduction label and removed workflowcanary label

mentioned in merge request !38410 (merged)