### Summary Project returns a 500 error when fetching the `dependencies` and `dependencies.json` endpoints. ### Steps to reproduce 1. Create a new project with the following files: <details><summary>.gitlab-ci.yml</summary> ```yaml dependency_scanning: script: - echo "test failing dependencies" artifacts: reports: dependency_scanning: gl-dependency-scanning-report.json license_scanning: script: - echo "test failing dependencies" artifacts: reports: license_scanning: gl-license-scanning-report.json ``` </details> <details><summary>gl-dependency-scanning-report.json</summary> ```json { "version": "15.0.0", "scan": { "analyzer": { "id": "gemnasium", "name": "Gemnasium", "url": "https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium", "vendor": { "name": "GitLab" }, "version": "4.0.2" }, "scanner": { "id": "gemnasium", "name": "Gemnasium", "url": "https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium", "vendor": { "name": "GitLab" }, "version": "4.0.2" }, "type": "dependency_scanning", "start_time": "2023-06-08T14:46:48", "end_time": "2023-06-08T14:46:53", "status": "success" }, "dependency_files": [ { "path": "BoloServices.JobFramework.Workflow.Activities/packages.lock.json", "package_manager": "nuget", "dependencies": [ { "iid": 13, "dependency_path": [ { "iid": 52 }, { "iid": 67 }, { "iid": 11 } ], "package": { "name": "Microsoft.AspNetCore.Http" }, "version": "2.1.1" }, { "iid": 11, "package": { "name": "NLog.Web.AspNetCore" }, "version": "4.9.0" }, { "iid": 67, "package": { "name": "RSA.seahorse.Logging" }, "version": "1.0.16" } ] } ], "vulnerabilities": [ { "id": "853e5bd2f40053a75ff373dcf1a4862db71cf621c08dbf253c5562c5ac75bb7f", "name": "Cookie parsing failure", "description": "A security feature bypass vulnerability exists in the way Microsoft ASP.NET Core parses encoded cookie names. The ASP.NET Core cookie parser decodes entire cookie strings which could allow a malicious attacker to set a second cookie with the name being percent encoded. The security update addresses the vulnerability by fixing the way the ASP.NET Core cookie parser handles encoded names., aka 'Microsoft ASP.NET Core Security Feature Bypass Vulnerability'.", "cve": "", "severity": "High", "solution": "Upgrade to version 2.1.22 or above.", "location": { "file": "BoloServices.JobFramework.Workflow.Activities/packages.lock.json", "dependency": { "iid": 13, "package": { "name": "Microsoft.AspNetCore.Http" }, "version": "2.1.1" } }, "identifiers": [ { "type": "gemnasium", "name": "Gemnasium-744de9f2-7edb-4e53-976c-d20777c420f8", "value": "744de9f2-7edb-4e53-976c-d20777c420f8", "url": "https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/nuget/Microsoft.AspNetCore.Http/CVE-2020-1045.yml" } ], "links": [ { "url": "https://github.com/advisories/GHSA-hxrm-9w7p-39cc" } ], "details": { "introduced_by_package": { "type": "text", "name": "Introduced by Package", "value": "BoloServices.jobframework:" }, "shortest_path": { "type": "list", "name": "Shortest Path", "items": [ { "type": "text", "value": "BoloServices.jobframework:" } ] }, "vulnerable_package": { "type": "text", "name": "Vulnerable Package", "value": "Microsoft.AspNetCore.Http:2.1.1" } } } ] } ``` </details> <details><summary>gl-license-scanning-report.json</summary> ```json { "version": "2.1", "licenses": [ { "id": "Apache-2.0", "name": "Apache License 2.0", "url": "https://opensource.org/licenses/Apache-2.0" } ], "dependencies": [ { "name": "AWS.Logger.AspNetCore", "version": "2.1.1", "package_manager": "nuget", "path": "Bolo.JobEngine.sln", "licenses": [ "https://aws.amazon.com/apache-2-0/" ] }, { "name": "AWS.Logger.Core", "version": "1.5.1", "package_manager": "nuget", "path": "Bolo.JobEngine.sln", "licenses": [ "https://aws.amazon.com/apache-2-0/" ] } ] } ``` </details> 2. Trigger a new pipeline on the default branch 3. Ping `http://<HOST>/<PROJECT_FULL_PATH>/-/dependencies.json` **The following error will be triggered**: ``` NoMethodError at /root/test-dependency/-/dependencies.json undefined method `fetch' for nil:NilClass next ancestor unless ancestor.fetch(:iid, false) # `block (2 levels) in augment_ancestors!': undefined method `fetch' for nil:NilClass ``` **The expected behaviour is the following**: ```json {"report":{"status":"ok","job_path":"/root/test-dependency/builds/514","generated_at":"2023-06-08T23:03:03.784Z"},"dependencies":[{"name":"Microsoft.AspNetCore.Http","packager":"C# (Nuget)","version":"2.1.1","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":[{"name":"RSA.Bolo.Logging","version":"1.0.16"},{"name":"NLog.Web.AspNetCore","version":"4.9.0"}]},"vulnerabilities":[{"name":"Cookie parsing failure","severity":"high","id":760,"url":"http://gdk.test:3000/root/test-dependency/-/security/vulnerabilities/760"}],"licenses":[]},{"name":"NLog.Web.AspNetCore","packager":"C# (Nuget)","version":"4.9.0","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":null},"vulnerabilities":[],"licenses":[]},{"name":"RSA.Bolo.Logging","packager":"C# (Nuget)","version":"1.0.16","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":null},"vulnerabilities":[],"licenses":[]}]} ``` ### Example Project [See internal note](#note_1421031368). Example project: https://gitlab.com/dharris/414559/ ### What is the current *bug* behavior? Accessing the `/-/dependencies/` page or `dependencies.json` from an impacted project gives a `500` error with exception `NoMethodError: undefined method 'fetch' for nil:NilClass`. ### What is the expected *correct* behavior? The user should be able to access project Dependencies information. ### Relevant logs and/or screenshots Kibana: https://log.gprd.gitlab.net/goto/ef6d2100-1126-11ee-a017-0d32180b1390 Stack trace: ``` ee/lib/gitlab/ci/reports/dependency_list/report.rb:72:in `block (2 levels) in augment_ancestors!', ee/lib/gitlab/ci/reports/dependency_list/report.rb:71:in `map!', ee/lib/gitlab/ci/reports/dependency_list/report.rb:71:in `block in augment_ancestors!', ee/lib/gitlab/ci/reports/dependency_list/report.rb:61:in `each_value', ee/lib/gitlab/ci/reports/dependency_list/report.rb:61:in `augment_ancestors!', ee/lib/gitlab/ci/reports/dependency_list/report.rb:19:in `dependencies', ee/lib/gitlab/ci/reports/dependency_list/report.rb:38:in `apply_license', ee/lib/gitlab/ci/parsers/security/dependency_list.rb:59:in `block in apply_licenses!', ee/lib/gitlab/ci/parsers/security/dependency_list.rb:58:in `each', ee/lib/gitlab/ci/parsers/security/dependency_list.rb:58:in `apply_licenses!', ee/app/models/ee/ci/pipeline.rb:138:in `block in dependency_list_report', ee/app/models/ee/ci/pipeline.rb:128:in `dependency_list_report', ee/app/services/security/dependency_list_service.rb:34:in `init_collection', ee/app/services/security/dependency_list_service.rb:23:in `execute', ee/lib/api/dependencies.rb:16:in `dependencies_by', ee/lib/api/dependencies.rb:48:in `block (2 levels) in <class:Dependencies>', ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call', ee/lib/gitlab/ip_address_state.rb:10:in `with', ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call', lib/api/api_guard.rb:219:in `call', ee/lib/omni_auth/strategies/group_saml.rb:41:in `other_phase', lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call', lib/gitlab/middleware/memory_report.rb:13:in `call', lib/gitlab/middleware/speedscope.rb:13:in `call', lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call', lib/gitlab/middleware/rails_queue_duration.rb:33:in `call', lib/gitlab/metrics/rack_middleware.rb:16:in `block in call', lib/gitlab/metrics/web_transaction.rb:46:in `run', lib/gitlab/metrics/rack_middleware.rb:16:in `call', lib/gitlab/jira/middleware.rb:19:in `call', lib/gitlab/middleware/go.rb:20:in `call', lib/gitlab/etag_caching/middleware.rb:21:in `call', lib/gitlab/middleware/query_analyzer.rb:11:in `block in call', lib/gitlab/database/query_analyzer.rb:37:in `within', lib/gitlab/middleware/query_analyzer.rb:11:in `call', lib/gitlab/middleware/multipart.rb:173:in `call', lib/gitlab/middleware/read_only/controller.rb:50:in `call', lib/gitlab/middleware/read_only.rb:18:in `call', lib/gitlab/middleware/same_site_cookies.rb:27:in `call', lib/gitlab/middleware/basic_health_check.rb:25:in `call', lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call', lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call', lib/gitlab/middleware/request_context.rb:15:in `call', lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call', config/initializers/fix_local_cache_middleware.rb:11:in `call', lib/gitlab/middleware/compressed_json.rb:45:in `call', lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call', lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call', lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call', lib/gitlab/middleware/release_env.rb:13:in `call' ``` ### Output of checks #### Results of GitLab environment info <!-- Input any relevant GitLab environment information if needed. --> GitLab.com version: GitLab Enterprise Edition 16.1.0-pre 9e2ea67bb3c #### Results of GitLab application Check <!-- Input any relevant GitLab application check information if needed. --> <details> <summary>Expand for output related to the GitLab application check</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:check SANITIZE=true`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`) (we will only investigate if the tests are passing) </pre> </details> ### Possible fixes The issue happens because `dependency_files[].dependencies[].dependency_path` (link to [security report schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/3b3b76e83722a97181f52f473a80f2f5713591e6/dist/dependency-scanning-report-format.json#L1010)) includes an `iid` which hasn't been defined. In the above example, the undefined `iid` is `52`. Those are some suggestions as possible fixes: 1. Update the analyser(s) in order to only allow existing `iid`'s as part of `dependency_path` 1. Update [parser](https://gitlab.com/gitlab-org/gitlab/-/blob/22843acb7d51f2578f3897416e4a2e23b77cb179/ee/lib/gitlab/ci/parsers/security/dependency_list.rb), [formatter](https://gitlab.com/gitlab-org/gitlab/-/blob/22843acb7d51f2578f3897416e4a2e23b77cb179/ee/lib/gitlab/ci/parsers/security/formatters/dependency_list.rb) and/or [report](https://gitlab.com/gitlab-org/gitlab/-/blob/22843acb7d51f2578f3897416e4a2e23b77cb179/ee/lib/gitlab/ci/reports/dependency_list/report.rb) so we can either consider this report as invalid or to just ignore the undefined dependency. Without looking at the bigger picture the following snippet would be a way of ignoring the undefined dependency: ```diff diff --git a/ee/lib/gitlab/ci/reports/dependency_list/report.rb b/ee/lib/gitlab/ci/reports/dependency_list/report.rb index 0cc769a5962e..fede268df9e2 100644 --- a/ee/lib/gitlab/ci/reports/dependency_list/report.rb +++ b/ee/lib/gitlab/ci/reports/dependency_list/report.rb @@ -72,7 +72,7 @@ def augment_ancestors! next ancestor unless ancestor.fetch(:iid, false) dependencies_by_iid[ancestor[:iid]] - end + end.compact! end end end (END) ``` 3. All the above ### Implementation plan - ~backend Ignore the undefined dependencies so in order to fix the 500 errors. The code snipped share [above](https://gitlab.com/gitlab-org/gitlab/-/issues/414559#possible-fixes) could be used as an initial approach.