Traversal ids queries

What does this MR do?

Closes #230456 (closed)

Currently to query the namespace hierarchy we must use recursive methods through Gitlab::ObjectHierarchy. We can query the namespace hierarchy a lot faster and easier if we store the path from the root ancestor to each namespace as an attribute on the namespace. We will store this on a new traversal_id attribute on Namespaces.

Say we have this Namespace hierarchy...

graph TD;
    gitlab-->backend;
    backend-->create;
    backend-->manage;
    create-->source;
    manage-->access;
    gitlab-->frontend;

Then the path from gitlab to access is gitlab / backend / manage / access. The path from gitlab to source is gitlab / backend / create / source.

We can use this structure for fast queries. If the current namespace is backend then all our descendants match the path gitlab / backend / *, all our ancestors match the path * / backend, and everyone in the same Namespace hierarchy matches gitlab / *.

We store that path using an array of Namespace ids in a new column on Namespace called traversal_ids.

The recursive Namespace query methods have been moved to a module at app/models/concerns/namespace/recursive_traversal.rb. Feature flags have been included to toggle between using recursive and linear querying methods.

Screenshots

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

doc/development/import_project.md

@@ -47,7 +47,9 @@ This method will take longer to import than the other methods and will depend on
 
 ### Importing via a Rake task
 
-[`import.rake`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/import_export/import.rake) was introduced for importing large GitLab project exports.
+> The [Rake task](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/tasks/gitlab/import_export/import.rake) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20724) in GitLab 12.6, replacing a GitLab.com Ruby script.
+
+This script was introduced in GitLab 12.6 for importing large GitLab project exports.
 
 As part of this script we also disable direct and background upload to avoid situations where a huge archive is being uploaded to GCS (while being inside a transaction, which can cause idle transaction timeouts).
 
@@ -63,9 +65,53 @@ Parameters:
 | `archive_path` | string | yes | Path to the exported project tarball you want to import |
 
 ```shell
-bundle exec rake "gitlab:import_export:import[root, root, testingprojectimport, /path/to/file.tar.gz]"
+bundle exec rake "gitlab:import_export:import[root, group/subgroup, testingprojectimport, /path/to/file.tar.gz]"
+```
+
+If you're running Omnibus, run the following Rake task:
+
+```shell
+gitlab-rake "gitlab:import_export:import[root, group/subgroup, testingprojectimport, /path/to/file.tar.gz]"
+```
+
+#### Troubleshooting
+
+Check the common errors listed below, what they mean, and how to fix them.
+
+##### `Exception: undefined method 'name' for nil:NilClass`
+
+The `username` is not valid.
+
+##### `Exception: undefined method 'full_path' for nil:NilClass`
+
+The `namespace_path` does not exist.
+For example, one of the groups or subgroups is mistyped or missing
+or you've specified the project name in the path.
+
+The task will only create the project.
+If you want to import it to a new group or subgroup then create it first.
+
+##### `Exception: No such file or directory @ rb_sysopen - (filename)`
+
+The specified project export file in `archive_path` is missing.
+
+##### `Name can contain only letters, digits, emojis ...`
+
+```plaintext
+Name can contain only letters, digits, emojis, '_', '.', dash, space. It must start with letter,
+digit, emoji or '_'. and Path can contain only letters, digits, '_', '-' and '.'. Cannot start
+with '-', end in '.git' or end in '.atom'
 ```
 
+The project name specified in `project_path` is not valid for one of the specified reasons.
+
+Only put the project name in `project_path`. If, for example, you put a path of subgroups in there
+it will fail with this error as `/` is not a valid character in a project name.
+
+##### `Name has already been taken and Path has already been taken`
+
+A project with that name already exists.
+
 ### Importing via the Rails console
 
 The last option is to import a project using a Rails console: