Skip to content

Use linear version ApplicationSettings#elasticsearch_limited_namespaces

What does this MR do?

In this method we're switching the behavior of the EE::ApplicationSettings#elasticsearch_limited_namespaces method to use the linear version. The new behavior is behind the linear_application_settings_elasticsearch_limited_namespaces feature flag.

How to setup and validate locally (strongly suggested)

  1. Enable the new method behavior feature flag
    Feature.enable(:linear_application_settings_elasticsearch_limited_namespaces)
  2. In rails console enable the traversal id feature flag
    Feature.enable(:use_traversal_ids)

SQL queries

The former sql query was:

WITH RECURSIVE base_and_descendants AS (
                                          (SELECT namespaces.*
                                           FROM namespaces
                                           WHERE namespaces.id IN
                                               (SELECT elasticsearch_indexed_namespaces.namespace_id
                                                FROM elasticsearch_indexed_namespaces))
                                        UNION
                                          (SELECT namespaces.*
                                           FROM namespaces,
                                                base_and_descendants
                                           WHERE namespaces.parent_id = base_and_descendants.id))
SELECT namespaces.*
FROM base_and_descendants AS namespaces

This is the query plan and the execution times are:

Time: 1.440 min
  - planning: 1.768 ms
  - execution: 1.440 min
    - I/O read: 1.621 min
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 764662 (~5.80 GiB) from the buffer pool
  - reads: 98668 (~770.80 MiB) from the OS file cache, including disk I/O
  - dirtied: 7840 (~61.30 MiB)
  - writes: 0

The new sql query is:

SELECT namespaces.*
FROM
  (SELECT DISTINCT on(namespaces.id) namespaces.*
   FROM namespaces,

     (SELECT namespaces.id
      FROM namespaces
      WHERE namespaces.id IN
          (SELECT elasticsearch_indexed_namespaces.namespace_id
           FROM elasticsearch_indexed_namespaces)) base
   WHERE (namespaces.traversal_ids @> ARRAY[base.id])) namespaces

This is the query plan and the execution times are:

Time: 1.141 min
  - planning: 0.825 ms
  - execution: 1.141 min
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 2768340 (~21.10 GiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Does this MR meet the acceptance criteria?

Conformity

Related to #339234 (closed)

Merge request reports