Optimistic lock for namespace descendants

What does this MR do and why?

Problem:

When we update namespace_descendants row for a given Group, the group and the cache record is locked for writing. For a large namespace hierarchy, this locking can take long time which affects group and project creation/update.

Solution:

This MR introduces a lock-free approach using optimistic locking.

  1. Namespaces::Descendant record has a field called outdated_at.
  2. This field is always bumped when the cache is marked outdated.
  3. In the Namespaces::UpdateDenormalizedDescendantsService we note down the current outdated_at value.
  4. We calculate the cached values without locks.
  5. When updating the Namespaces::Descendant record with the new cache values we compare the old outdated_at value with the current outdated_at value from the DB.
  6. If the timestamp differ, the cache stays in outdated state so the next run can attempt caching it again.
  7. If the timestamp is the same that means that the group hierarchy did not change during the cache calculation. The record will be marked up to date.

The UPDATE query uses low lock timeout to avoid lock contention.

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Adam Hegyi

Merge request reports

Loading