# Configurable scan profiles ## TL;DR Introduce a way to configure security scan profiles at the trigger level: per trigger, or shared across several triggers, instead of a single configuration per profile. - Configuration is stored in a new table `security_scan_profile_configurations`. Each trigger points to it via a nullable FK. - Many triggers can share one configuration row and each trigger may also have its own. A `NULL` FK means "use defaults". - **Store lean, read fat**: persist only the diff vs. code-defined defaults, then serve a deep-merged configuration with defaults on read. - Configuration jsonb is schema-validated against code-defined JSON schemas keyed by `scan_type`. - The existing and unused per-profile column `security_scan_profiles.configuration` is removed. ## Assumptions - Configuration is owned and resolved per trigger, not per profile. We deliberately keep it at this higher level and avoid a separate configuration per scanner. - Schema and defaults are keyed by `scan_type`. - The existing `configuration` column is empty, so dropping it needs no backfill. - Defaults live in code (`Security::ScanProfiles::Configuration.defaults_for`), so adding a new default later needs no backfill of existing rows. - A `NULL` FK resolves to defaults, such that no row is written until a customization exists. ## DB structure ```mermaid erDiagram namespaces ||--o{ security_scan_profiles : "has many" security_scan_profiles ||--o{ security_scan_profile_triggers : "has many" security_scan_profiles ||--o{ security_scan_profile_configurations : "has many" security_scan_profile_configurations |o--o{ security_scan_profile_triggers : "shared by (0..N triggers)" security_scan_profile_configurations { bigint id PK bigint security_scan_profile_id FK bigint namespace_id jsonb configuration "stored diff vs defaults" smallint configuration_version } security_scan_profile_triggers { bigint security_scan_profile_configuration_id FK "nullable, NULL = defaults" } ``` ## DB changes Two changes, both on the `sec_db`: - New `security_scan_profile_configurations` table: - `id`: PK - `security_scan_profile_id`: `bigint`, hard FK to `security_scan_profiles` (`ON DELETE CASCADE`), indexed - `namespace_id`: `bigint`, loose FK to `namespaces` (cross-schema, mirror sibling tables), indexed - `configuration`: `jsonb DEFAULT '{}' NOT NULL` - `configuration_version`: `smallint` for versioning - `created_at`, `updated_at` - New nullable column `security_scan_profile_configuration_id` on `security_scan_profile_triggers` - FK to the new table (`ON DELETE SET NULL`) so deleting a config resets its triggers to defaults. - **Drop the old column** - Remove `security_scan_profiles.configuration`. Because it is already deployed on use the two-step path: add `ignore_columns` and remove code references, then `remove_column` in a post-deployment migration in a follow-up release. There is no way to set values to this column today. ## App-level changes ### `Security::ScanProfiles::Configuration` Today this is a simple class holding the per-`scan_type` defaults and the merge logic. We promote it to the AR model for the new table, keeping its name, its `defaults_for`, and its `Defaults` submodule. It owns the stored row, its validation, and the defaults lookup. Resolution (merging defaults with a trigger's diff) moves to `Security::ScanProfile` (below). ```ruby class Security::ScanProfiles::Configuration < ::SecApplicationRecord self.table_name = 'security_scan_profile_configurations' belongs_to :scan_profile, foreign_key: :security_scan_profile_id, inverse_of: :configurations belongs_to :namespace, optional: false # The FK lives on the trigger, so one configuration row can be shared by many triggers. has_many :scan_profile_triggers, foreign_key: :security_scan_profile_configuration_id, inverse_of: :configuration # json_schema validation keyed by scan_profile.scan_type (size_limit TBD, see open questions) # defaults_for(scan_type) and the Defaults submodule stay here. end ``` ### `Security::ScanProfileTrigger` A trigger delegates to its profile to resolve its effective configuration. ```ruby belongs_to :configuration, class_name: 'Security::ScanProfiles::Configuration', foreign_key: :security_scan_profile_configuration_id, optional: true def effective_configuration scan_profile.configuration_for(self) end ``` ### `Security::ScanProfile` Owns resolution, since it knows the `scan_type` and its defaults. Also remove the old `configuration` validations, the old `effective_configuration`, and the post-processing schema helpers (they move to the config model), and add `ignore_columns :configuration`. ```ruby def configuration_for(trigger) Security::ScanProfiles::Configuration.defaults_for(scan_type) .deep_merge((trigger.configuration&.configuration || {}).deep_symbolize_keys) end ``` ### `Security::ScanProfiles::SetConfigurationService` Single write path for set / update / reset. Signature: `new(trigger, values:, current_user:)` (see "When writing"). ### Authorization Unchanged from today: reuse `apply_security_scan_profiles` for writes and `read_security_scan_profiles` for reads. ## When reading `trigger.effective_configuration` delegates to `scan_profile.configuration_for(trigger)`: - `NULL` FK: return `defaults_for(scan_type)`. - Otherwise: return `defaults_for(scan_type)` deep-merged with the stored diff. ## When writing `SetConfigurationService(trigger, values:, current_user:)`: - Persist `values` as given (store lean). - Copy-on-write: - if the trigger's config is referenced only by this trigger, update in place. - if it is shared or `NULL`, create a new row and re-point this trigger so siblings are untouched. - Reset: blank `values` set the trigger's FK to `NULL`, falling back to defaults. - Run orphan GC (below) in the same transaction. ## Orphan config garbage collection Re-pointing a trigger or resetting one can leave a config row referenced by zero triggers. To fix this, we should delete config rows with no referencing triggers, in the same transaction as the write, using a NULL-safe anti-join. Example: ```ruby profile.configurations .where('NOT EXISTS (SELECT 1 FROM security_scan_profile_triggers t ' \ 'WHERE t.security_scan_profile_configuration_id = ' \ 'security_scan_profile_configurations.id)') .delete_all ``` The set is bounded to one profile's rows. The `ON DELETE CASCADE` from the profile still cleans everything when a profile is destroyed, and `ON DELETE SET NULL` on the trigger FK keeps triggers valid if a config is removed. ## Open questions 1. **JSON size limit**: We do not yet know the expected config size. Per our [docs](https://docs.gitlab.com/development/migration_style_guide/#storing-json-in-database), a JSONB column should use `JsonSchemaValidator` with a `size_limit`, with `64KB` as the recommended maximum. Since we are uncertain about the config size, we may start with that `64KB` limit at the application level only for flexibility and revisit later. 2. **Strip values vs. store some defaults**: strip keys equal to the default before persisting or store whatever the caller submits? Product decision. affects how lean storage really is and how values are shown in the UI. 3. **Dedup on write**: should the service auto-detect an existing identical config row for the profile and re-point to it (producing the shared object), or is sharing always driven by an explicit action? 4. **GraphQL structure**: TBD. The goal is to have strictly typed configurations types based on the profile. Mutations should be relatively straightforward in the initial stage.