Ghost pipelines
We would benefit from having pipelines for internal usage only. They would differ from regular pipelines in various ways:
- They can't be triggered by users
- They would never be visible to users, nor notify them
- They would not use users minutes
- They would have their own API
- They can be converted to regular pipelines if needed
These hidden builds would be leveraged by the Secure team in the first place, but I'm sure we'll find more usages in the future for different stages. Ghost pipelines would be used to:
- test updates during auto-remediation: While we're already able to provide patches and create a Merge Request when a vulnerability is found (under certain conditions), there's no guarantee that this update will not break the app. We rely on the user judgment, who will rely in turn on the MR pipeline. It's a good MVC, but it can be very noisy if we create a lot of MRs which will fail eventually.
- maybe run security tests during live coding with the WebIDE. I don't think it's useful to keep any pipeline other than the latest, as we might have a lot of them.
- QA on analyzers updates: We only have a few test-projects, and they are too simple to catch real-world issues. We need to be more prepare for large customers having thousands of heterogeneous projects. We can leverage the data available in Gitlab.com, and run pre-versions of our analyzers on these projects, without disturbing at any time our users and customers. Of course, we would only use public projects. By doing so, we would be able to validate updates of our analyzers, measure different metrics like:
- Number of supported project out-of-the-box (without any specific setup)
- Run time
- Number and quality of findings
- validate new analyzers, instead of just testing them with a basic project before releasing to our entire user-base.
- debug user issues if their project can't be forked for some reason.
- notify users when new vulnerabilities are discovered
Edited by Philippe Lafoucrière