Technical Discovery: Migrate away from user direct editing in pipeline yml for secure analyzers

Goal

Instant, non MR needed, API-able (CRUD) analyzer/job data at project, group, and instance level

Data: is an analyzer disabled/enabled? configuration?

Desired adds:

  • limit who can update/change at a more granular level
  • dynamically set above by label, group, instance
  • force it to apply and be unavoidable

Complexities:

  • how to handle approval groups at rollup?
  • how to handle when 2 apply?
  • this MUST log audit events

Nicole's idea

Could we store settings for an analyzer in a database, then when a pipeline runs call an api that dynamically builds the yml based off the database using https://docs.gitlab.com/ee/ci/parent_child_pipelines.html#dynamic-child-pipelines somehow?

I am welcome to any ideas

Why? / What do we get

  • Today we are VERY bad at quickly determining the analyzer status (enabled/disabled) at a project level as we rely on if and only if it was run last pipeline - we should be able to instantly query if a secure category applies to a project.
  • Users want to quickly, for thousands of projects, see what analyzers are set to be enabled and configured, and This would make it easier to tell if a thing is enabled, or not, at a summary (group/instance) level quickly (i.e. if they enabled by labels or for all in a group)
  • People which to bulk apply things instantly and in bulk to 1 or more project. For example setup ~"Category:Dependency Scanning" configuration and then choose for it to apply to 1 or more projects with as few clicks and wait as possible. Users want to apply things conditionally that can change - perhaps leveraging a specific tag/label, everything in a group, everything in the instance, or only specific selected projects. (this is a top customer request)
  • users which to share rules (allow lists for example) across many projects
  • Avoid the need for an MR (wait, cost, perceived complexity)
  • It would make changes instantly apply to all future run pipelines instead of having to worry about which will or will not get new config. peace of mind they set the security rules and it instantly applies.
  • It would allow users who love APIs to query (and maybe eventually update) via API
  • allow more restrictions of who may edit security analyzers (i.e. disable them)
  • allow more dynamic conditionals of what jobs and settings should apply /run

Considerations

would need to coordinate with secure and protect, pipeline team, autodevops and compliance to make sure we could also force include, and not break any of their things and that everyone is on board with our POC as a end goal.

related epics / issues

some pre-requisties:

  • research on child pipelines #283955 (closed)
  • the backlog item to add a variable to have child artifacts able to be passed up &4019
  • variable passing &4529
Edited by Nicole Schwartz