Spike: Implement Experimental Features Mechanism in policy.yml for Security Policies

Proposal

Problem Statement

We currently lack a consistent and efficient mechanism for introducing new experimental features in the Security Policies domain. Our current approaches have several drawbacks:

  1. Building frontend toggles in Project/Group settings is time-consuming.
  2. Creating new application/group settings in the database is resource-intensive.
  3. Using feature flags for long-lived experiments goes against GitLab's best practices.

Proposed Solution

Introduce a new experiments field in the policy.yml file. This field will be an object containing experiment configurations, allowing users to enable and configure experimental features directly in their security policy project repository.

Example structure:

approval_policies: []
scan_execution_policy: []
experiments:
  scheduled_pipeline_execution_policies:
    enabled: true
    configuration:
      min_time_window: 10000
  limits_based_on_scope:
    enabled: true
    configuration:
      max_applied_policies: 10
  other_experiment:
    enabled: false

Benefits

  1. Simplified user experience: Users can easily enable and configure experimental features.
  2. Reduced development overhead: No need for frontend toggles or database settings for each experiment.
  3. Adheres to GitLab best practices: Avoids long-lived feature flags.
  4. Flexible configuration: Allows for experiment-specific configuration options.
  5. Easy evaluation: Users can test experimental features before GA release without waiting for feature flag enablement.

Implementation Considerations

  1. Define a standard structure for experiment configurations in policy.yml.
  2. Implement a parser to read and validate the experiments section.
  3. Integrate the experimental feature toggle mechanism with the existing security policy enforcement logic.
  4. Ensure proper documentation for users on how to utilize the new experiments field.
  5. Develop a process for graduating experimental features to stable features.

Next Steps:

  1. Refine and approve the proposed solution.
  2. Break down the implementation into smaller, actionable tasks.
  3. Prioritize the work within the Security Policies team's roadmap.
  4. Assign team members to begin implementation.