Enhance performance testing infrastructure

Background

Our current performance testing infrastructure needs enhancement to support better testing Security Policies with large-scale groups and projects. We need to increase our test coverage beyond the current 4,000 project limit.

Objective

Create comprehensive performance testing infrastructure for Security Policies to validate functionality and performance at scale.

Requirements

Test Environment Enhancement

  • Increase test group size beyond current 4,000 projects limit
    • Create test group with 20,000 projects
    • Create test group with 50,000 projects
    • Configure projects with representative Git data
  • Set up automated infrastructure for group/project creation
  • Configure monitoring for test environment

Performance Test Scenarios

  • Implement test scenarios for:
    • Policy creation and updates
    • Policy synchronization across large groups
    • Policy linking/unlinking operations
    • Policy scope changes
    • Multiple policy interactions

Test Data Generation

  • Create tools for generating:
    • Test groups with configurable sizes
    • Projects with realistic structures
    • Representative policy configurations
    • Sample merge requests and pipelines

Automated Testing Framework

  • Develop automated tests for:
    • Query performance validation
    • Policy synchronization timing
    • Database operation efficiency
    • Worker performance
  • Set up performance benchmarks
  • Configure test result collection and reporting

CI Integration

  • Add performance tests to CI pipeline
  • Set up performance regression detection
  • Configure automated reporting of test results
  • Add performance test coverage to MR requirements

Acceptance Criteria

  1. Performance testing environment supports groups with 50,000+ projects
  2. Automated test suite covers all critical policy operations
  3. CI pipeline includes performance tests
  4. Performance regression detection is operational
  5. Documentation updated with:
    • Test environment setup
    • Test scenario descriptions
    • Performance benchmarks
    • Regression thresholds

Technical Details

Test Scenarios to Implement

# Policy Operations
- Create policy for large group
- Update existing policy
- Link/unlink security policy project
- Modify policy scope
- Policy synchronization

# Database Operations
- Project ID fetching
- Policy index updates
- Policy rule synchronization
- Background worker operations
Performance Metrics to Track
rubyCopy# Timing Metrics
- Policy creation time
- Synchronization duration
- Query execution time
- Worker processing time

# Resource Usage
- Database load
- Memory consumption
- CPU utilization
- 
/label ~"devops::security risk management" ~"group::security policies" ~performance ~testing 
/copy_metadata #373191
Edited by Alan (Maciej) Paruszewski