Skip to content

Implement a namespace hierarchy iterator library

What does this MR do and why?

This change implements an efficient iterator over the namespace hierarchy graph in SQL based on the PoC documented here: https://docs.gitlab.com/ee/development/database/poc_tree_iterator.html

Note: the query is quite complex so I tried to not overdo Arel and used raw SQL when it made sense.

Queries:

How to set up and validate locally

Snippet for listing the tree for a group:

group = Group.first
cursor = { current_id: group.id, depth: [group.id] }
iterator = iterator = Gitlab::Database::NamespaceEachBatch.new(namespace_class: Namespace, cursor: cursor)
iterator.each_batch(of: 100) do |ids|
  puts ids
end

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #433479 (closed)

Edited by Adam Hegyi

Merge request reports