Skip to content

Add `pre_order` option to EachBatch

Mehmet Emin INAC requested to merge add_pre_order_option_to_each_batch into master

What does this MR do and why?

This MR introduces an option for the EachBatch module to iterate over the records by using multiple columns.

The old SQL query generated by `EachBatch`
SELECT
    "security_findings"."id"
FROM
    "security_findings"
WHERE
    "security_findings"."scan_id" BETWEEN 19195974 AND 19196979
    AND "security_findings"."id" >= 1949656930
ORDER BY
    "security_findings"."id" ASC
LIMIT 1 OFFSET 1000
The new SQL query generated with this option
EXPLAIN
SELECT
    "security_findings"."scan_id",
    "security_findings"."id"
FROM
    "security_findings"
WHERE
    "security_findings"."scan_id" BETWEEN 19195974 AND 19196979
    AND ((security_findings.scan_id,
            security_findings.id) >= (19195975,
            1949656930))
ORDER BY
    "security_findings"."scan_id" ASC,
    "security_findings"."id" ASC
LIMIT 1 OFFSET 1000

Extracted from Mark stale `security_scans` as `purged` (!82711 - merged).

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mehmet Emin INAC

Merge request reports