Skip to content

Improve testing to anticipate possible bottlenecks on production DB/queue

Why?

Action item from FCL for incident 8619.

Definition of Done

  1. Create a proposal on how could we test and anticipate the impact of the introduced changes on DB/queue usage potentially could degrade performance.
  2. Introduce a proof of concept based on introduced and rollbacked MR: gitlab!114426 (merged)

Propose a data testing approach and explore potential options:

  1. Use real-world data: Use real-world data that accurately reflects the usage patterns of the system. This can be achieved by capturing and anonymizing production data for use in testing.
  2. Use synthetic data: Create synthetic data that simulates real-world usage patterns. This can be useful in identifying edge cases and unusual usage patterns.
  3. Use randomized data: Use randomized data to test the robustness of the system. This can be useful in identifying potential security vulnerabilities and unexpected behavior.
  4. Use database profiling tools: Use database profiling tools to measure the impact of changes on the database. These tools can provide detailed information on query performance and resource usage.