Better support scanning of multiple container images
<!--The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution.-->
### Release notes
<!--What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). "-->
### Problem to solve
1. As a developer, I build multiple images in my pipeline and want to easily scan all of them for vulnerabilities.
1. As an application security engineer, I want to easily scan multiple images in an internal (GitLab hosted) or ~external~ container registry for vulnerabilities with minimal configuration.
1. As an application security engineer, I want to have access to individual security reports generated by each scanned image (https://gitlab.com/groups/gitlab-org/-/epics/3139#note_2529670552).
- Also required if feature will be available for Free and Premium tiers.
### Intended users
* [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager)
* [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead)
* [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer)
* [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst)
* [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer)
### Proposal
<!--How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey-->
1. Users will be able to pass in an array of images to be scanned as part of ~a single~ many Container Scanning ~job~ jobs.
2. Users will be able to enter a regular expression where all images in the registry that match that regular expression are scanned in **multiple concurrent** jobs. --> **To be moved to a different epic**
3. ~~Users will be able to have a single Container Scanning template automatically scan all images that are pushed to a registry as part of a pipeline. This should happen without needing to specify each of the image names. This will happen even when the pipeline builds multiple images. There must be a way to override this behavior.~~
4. ~~Users will be able to filter the Vulnerability Report page by Image name.~~
5. ~~Users will be able to filter the Security tab of the Pipeline page by Image name.~~
Edit to number 4 and 5 above: ~"group::composition analysis" is no longer chartered to deal in front end work. This will be passed to a front-end focused team. Our UX needs are being tracked in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/15842) (format is still a WIP).
Note: This proposal does **not** add support for multiple registries. Support will remain limited to just one registry per container_scanning CI job.
### Design
This feature will use dynamic pipelines to configure multi-image scanning.

<details><summary>Proposals</summary>
#### Parallel Matrix enabled CI/CD template
**Pros**
- Scans defined images quickly and in parallel
- Multi-registry authentication reuses `CS_REGISTRY_USER` and `CS_REGISTRY_PASSWORD`
- Can upload multiple container-scanning reports
**Cons**
- Inherits all of the [parallel:matrix](https://docs.gitlab.com/ci/yaml/#parallelmatrix) edge cases. These include things like:
- Names can at most be 255 characters. Even defining a CS_IMAGE var with `registry.gitlab.com/gitlab-org/security-products/analyzers/container-scanning/tmp:12345678` puts us at 112 chars. Any other config values will cause this to grow.
{width=25%}
- Job names can become confusing.
- Is this a concern?
- Limited to at most 200 images.
- Is this an issue? How would we know if we're getting close to this limit, or if it's breached? Lots of uncertainty.
- Inputs can become very cumbersome to maintain
#### Single Job Many Images
**Pros**
- Easy to find job in pipeline page and job page
- Easier to interface with
**Cons**
- Can only scan one authenticated registry at a time until we refactor the analyzer
- Does not scale well. Multiple images in a series can make for _very_ long scan times.
- I think Trivy already does things in parallel, so it wouldn't be very beneficial for us to just launch N number of Trivy scans at once.
- Requires us to merge together all of the findings into a singular report
- One image scan failure will cause entire job to fail unless we handle this gracefully
- How much can we mitigate this?
#### Create CI/CD config from container scanning config
This solution involves a CLI or library that takes in a container scanning configuration, and returns a CI/CD that they include in their template, or is used as a [dynamic pipeline].
**Pros**
- We can customize the jobs names to be human readable
- Establishes contract on what we expect as input. Makes migrations easy for users that use this interface.
**Cons**
- Job names must still be 255 characters at most
- Might not play well with our move to configuration profiles
- Inherits limitations of a [dynamic pipeline]
</details>
### Documentation
<!--See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change
* Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements
* If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html-->
Documentation will be updated to describe how to use these features.
### Availability & Testing
<!--This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier.
What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing?
Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance.
* Unit test changes
* Integration test changes
* End-to-end test change
See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning-->
- We should test that we don't overwrite files or cause random results if using a shared runner (https://gitlab.com/groups/gitlab-org/-/epics/3139#note_2530788465)
- Test against docker
- Test against containerd
- Test against podman
### What does success look like, and how can we measure that?
<!--Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this.-->
### What is the type of buyer?
<!--What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/
In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers-->
~"GitLab Ultimate"
### Is this a cross-stage feature?
<!--Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features-->
### Links / references
<!--Label reminders - you should have one of each of the following labels if you can figure out the correct ones-->
[dynamic pipeline]: https://docs.gitlab.com/ci/pipelines/downstream_pipelines/#dynamic-child-pipelines
<!--triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION-->
_This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc._
<!--triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION-->
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
epic