Add execution fallback pipeline for ES to PG filter handling
What does this MR do and why?
Introduce ExecutionFallback::Executor, a shared abstraction for implementing Elasticsearch to Postgres execution fallback for search filters.
This executor:
- Splits declared fallback filters from native Elasticsearch query arguments
- Activates fallback only when the
elastic_filters_execution_fallbackfeature flag is enabled - Over-fetches results to preserve pagination density
- Supports both cursor (
first/last) and offset (page/per_page) pagination - Runs enrichment queries in bounded batches
- Applies fallback filtering
- Trims results back to the originally requested window
- Enforces a configurable safety cap on maximum records processed
It standardises fallback behaviour across search resolvers and provides a safe, bounded, and testable framework for gradually introducing filters before Elasticsearch-native support is available.
The Enrichments::Composer builds an ordered registry of enrichment modules (ENRICHMENTS), while the executor resolves and orchestrates them explicitly.
This allows:
- Validating filter logic before full ES rollout
- Safely enabling functionality via an ops feature flag
- Testing search behaviour without requiring Elasticsearch implementation
The executor ensures no behavioural change when the feature flag is disabled and maintains pagination correctness while protecting performance through controlled overfetch and batching.
References
- Issue: Introduce ES Execution Fallback Base Framework (#586566)
- Epic: Phase 2: Execution Fallback for Parallel Develo... (gitlab-org#20523)
This work is split into three MRs to make adoption incremental and easier to review:
- This first MR introduces the execution fallback base framework.
- The second MR demonstrates how to integrate it into a GraphQL resolver: Draft: Add execution fallback support to Vulner... (!219313)
- The third MR validates the approach by providing a concrete example of a fallback filter implementation: Draft: Enable false positive execution fallback... (!219317).
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
Usage: define an enrichment module
Declare which filters require fallback in the pre-loader:
# frozen_string_literal: true
module Search
module Elastic
module Preloaders
module Vulnerability
class FalsePositive < Base
extend Search::Elastic::Filters::ExecutionFallback::Composer
fallback_filter :false_positive
def perform_preload; end
# Optional
# def normalize_boolean(value)
# value.map { |v| ActiveModel::Type::Boolean.new.cast(v) }
# end
# Registers :false_positive as a fallback filter.
#
# - :false_positive - filter argument key expected in search params
#
# - method: (default: same as :perform_preload)
# Optional instance method executed with a batch of record IDs.
#
# - normalize_with: (default: none)
# Optional instance method used to normalize user-provided
# filter values before comparison.
# If omitted and an instance method `normalize_filter` exists,
# it will be used automatically.
#
# fallback_filter :false_positive, method: :perform_preload, normalize_with: :normalize_boolean
end
end
end
end
Before usage
Usage in resolver
Search::Elastic::Filters::ExecutionFallback::Executor.fetch_with_fallback(
args,
user: current_user,
namespace: Search::Elastic::Preloaders::Vulnerability
) do |query_args|
fetch_vulnerabilities_from_elasticsearch(query_args)
end
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.