Dependency List and gets results from unfinished pipeline
release bug note gitlab-com/www-gitlab-com!85857 (merged)
Summary
Recently we changed behaviour of dependency_scanning
analyzer so it generates multiple test jobs. Due to how we fetch pipeline to get results for Dependency List and License List pages pipeline query and service used in endpoints we might end in a situation when one of dependency_scanning
jobs is finished even if the pipeline is still running, we can get wrong results in Dependency List page as only part of dependencies will be displayed.
Additional drawback of the current solution is the fact that for large pipelines we won't get vulnerabilities in the Dependency List until the pipeline is finished (as we get vulnerabilities from the same pipeline as dependencies information (dependency_files
section of the report).
Note: Related API endpoint is also affected, as it also uses ReportFetchService
to get a pipeline.
Note: Licence list page isn't affected because license_scanning
is running as one job.
Steps to reproduce
The easiest way to reproduce it is to go to https://gitlab.com/gitlab-org/gitlab/-/dependencies and wait until the next pipeline on master
starts. And then wait until one of the dependency_scanning
jobs finished, and observe the page.
Example Project
https://gitlab.com/gitlab-org/gitlab/-/dependencies
What is the current bug behavior?
dependencies displayed partially only for the finished job.
What is the expected correct behavior?
We need to use the different pipeline to fetch dependencies: only one that has passed and contains the right job artifacts.
Relevant logs and/or screenshots
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
GitLab 13.11 - latest version
Possible fixes
Update https://gitlab.com/gitlab-org/gitlab/-/blob/998fe1103b67075ec00c42150287513c1563785e/ee/app/services/security/report_fetch_service.rb#L14 but we need to take into account the fact that some projects (as GitLab itself) can have multiple pipeline configuration. We need to get a finished pipeline with existing dependency scanning artifacts
- add tests to reproduce this bug
- update https://gitlab.com/gitlab-org/gitlab/-/blob/75173a799bd350f52dcd3c0b646243ed3d793b4c/ee/app/models/ee/project.rb#L316 to include
success
scope:all_pipelines.success.newest_first(ref: project.default_branch).with_reports