Add execution fallback pipeline for ES to PG filter handling

What does this MR do and why?

Introduce ExecutionFallback::Executor, a shared abstraction for implementing Elasticsearch to Postgres execution fallback for search filters.

This executor:

  • Splits declared fallback filters from native Elasticsearch query arguments
  • Activates fallback only when the elastic_filters_execution_fallback feature flag is enabled
  • Over-fetches results to preserve pagination density
  • Supports both cursor (first/last) and offset (page/per_page) pagination
  • Runs enrichment queries in bounded batches
  • Applies fallback filtering
  • Trims results back to the originally requested window
  • Enforces a configurable safety cap on maximum records processed

It standardises fallback behaviour across search resolvers and provides a safe, bounded, and testable framework for gradually introducing filters before Elasticsearch-native support is available.

The Enrichments::Composer builds an ordered registry of enrichment modules (ENRICHMENTS), while the executor resolves and orchestrates them explicitly.

This allows:

  • Validating filter logic before full ES rollout
  • Safely enabling functionality via an ops feature flag
  • Testing search behaviour without requiring Elasticsearch implementation

The executor ensures no behavioural change when the feature flag is disabled and maintains pagination correctness while protecting performance through controlled overfetch and batching.

References

This work is split into three MRs to make adoption incremental and easier to review:

Screenshots or screen recordings

Before After

How to set up and validate locally

Usage: define an enrichment module

Declare which filters require fallback in the pre-loader:

# frozen_string_literal: true

module Search
  module Elastic
    module Preloaders
      module Vulnerability
      class FalsePositive < Base
        extend Search::Elastic::Filters::ExecutionFallback::Composer

        fallback_filter :false_positive

        def perform_preload; end

        # Optional
        # def normalize_boolean(value)
        #   value.map { |v| ActiveModel::Type::Boolean.new.cast(v) }
        # end

        # Registers :false_positive as a fallback filter.
        #
        # - :false_positive - filter argument key expected in search params
        #
        # - method: (default: same as :perform_preload)
        #     Optional instance method executed with a batch of record IDs.
        #
        # - normalize_with: (default: none)
        #     Optional instance method used to normalize user-provided
        #     filter values before comparison.
        #     If omitted and an instance method `normalize_filter` exists,
        #     it will be used automatically.
        #
        # fallback_filter :false_positive, method: :perform_preload, normalize_with: :normalize_boolean
      end
    end
  end
end

Before usage

Usage in resolver

Search::Elastic::Filters::ExecutionFallback::Executor.fetch_with_fallback(
  args,
  user: current_user,
  namespace: Search::Elastic::Preloaders::Vulnerability
) do |query_args|
  fetch_vulnerabilities_from_elasticsearch(query_args)
end

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ugo Nnanna Okeadu

Merge request reports

Loading