Improve performance of Members::ExpiringWorker

What does this MR do and why?

Improve performance of Members::ExpiringWorker

We are seeing PG::QueryCanceled on production due to inefficient batch queries in this worker. This change improves performance on the main batch query so we hopefully will not see these timeouts.

References

Database Review

It is a little hard to confirm that this will fix the issue on production. The existing query performs badly on a cold cache - about 4 seconds to get a batch 🙀 However against a hot cache, this can go down to as little as 22ms. Tweaking the expires_at date limits doesn't change much of the buffer hits vs disk reads in the query stats, so it's hard to judge which changes affect the actual time performance of the query.

Changing the ORDER BY clause to use the same column that is indexed for the query - expires_at reduces the cost= part of the results output pretty dramatically. From cost=8372.97..10425.06 to cost=1.00..2825.06 . So I think we will see significant improvement with this measure.

Current production query - cold cache - Postgres.ai link

Current production query - hot cache - Postgres.ai link

New query - Postgres.ai link

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. Run Members::ExpiringWorker.new.perform on Rails console
  2. Inspect generated queries to ensure they are using expires_at for the ORDER BY clause

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #510808 (closed)

Edited by Andrew Evans

Merge request reports

Loading