Accessing Dependencies.json results in a 500 error
Summary
Project returns a 500 error when fetching the dependencies and dependencies.json endpoints.
Steps to reproduce
- Create a new project with the following files:
.gitlab-ci.yml
dependency_scanning:
script:
- echo "test failing dependencies"
artifacts:
reports:
dependency_scanning: gl-dependency-scanning-report.json
license_scanning:
script:
- echo "test failing dependencies"
artifacts:
reports:
license_scanning: gl-license-scanning-report.jsongl-dependency-scanning-report.json
{
"version": "15.0.0",
"scan": {
"analyzer": {
"id": "gemnasium",
"name": "Gemnasium",
"url": "https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium",
"vendor": {
"name": "GitLab"
},
"version": "4.0.2"
},
"scanner": {
"id": "gemnasium",
"name": "Gemnasium",
"url": "https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium",
"vendor": {
"name": "GitLab"
},
"version": "4.0.2"
},
"type": "dependency_scanning",
"start_time": "2023-06-08T14:46:48",
"end_time": "2023-06-08T14:46:53",
"status": "success"
},
"dependency_files": [
{
"path": "BoloServices.JobFramework.Workflow.Activities/packages.lock.json",
"package_manager": "nuget",
"dependencies": [
{
"iid": 13,
"dependency_path": [
{
"iid": 52
},
{
"iid": 67
},
{
"iid": 11
}
],
"package": {
"name": "Microsoft.AspNetCore.Http"
},
"version": "2.1.1"
},
{
"iid": 11,
"package": {
"name": "NLog.Web.AspNetCore"
},
"version": "4.9.0"
},
{
"iid": 67,
"package": {
"name": "RSA.seahorse.Logging"
},
"version": "1.0.16"
}
]
}
],
"vulnerabilities": [
{
"id": "853e5bd2f40053a75ff373dcf1a4862db71cf621c08dbf253c5562c5ac75bb7f",
"name": "Cookie parsing failure",
"description": "A security feature bypass vulnerability exists in the way Microsoft ASP.NET Core parses encoded cookie names. The ASP.NET Core cookie parser decodes entire cookie strings which could allow a malicious attacker to set a second cookie with the name being percent encoded. The security update addresses the vulnerability by fixing the way the ASP.NET Core cookie parser handles encoded names., aka 'Microsoft ASP.NET Core Security Feature Bypass Vulnerability'.",
"cve": "",
"severity": "High",
"solution": "Upgrade to version 2.1.22 or above.",
"location": {
"file": "BoloServices.JobFramework.Workflow.Activities/packages.lock.json",
"dependency": {
"iid": 13,
"package": {
"name": "Microsoft.AspNetCore.Http"
},
"version": "2.1.1"
}
},
"identifiers": [
{
"type": "gemnasium",
"name": "Gemnasium-744de9f2-7edb-4e53-976c-d20777c420f8",
"value": "744de9f2-7edb-4e53-976c-d20777c420f8",
"url": "https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/nuget/Microsoft.AspNetCore.Http/CVE-2020-1045.yml"
}
],
"links": [
{
"url": "https://github.com/advisories/GHSA-hxrm-9w7p-39cc"
}
],
"details": {
"introduced_by_package": {
"type": "text",
"name": "Introduced by Package",
"value": "BoloServices.jobframework:"
},
"shortest_path": {
"type": "list",
"name": "Shortest Path",
"items": [
{
"type": "text",
"value": "BoloServices.jobframework:"
}
]
},
"vulnerable_package": {
"type": "text",
"name": "Vulnerable Package",
"value": "Microsoft.AspNetCore.Http:2.1.1"
}
}
}
]
}gl-license-scanning-report.json
{
"version": "2.1",
"licenses": [
{
"id": "Apache-2.0",
"name": "Apache License 2.0",
"url": "https://opensource.org/licenses/Apache-2.0"
}
],
"dependencies": [
{
"name": "AWS.Logger.AspNetCore",
"version": "2.1.1",
"package_manager": "nuget",
"path": "Bolo.JobEngine.sln",
"licenses": [
"https://aws.amazon.com/apache-2-0/"
]
},
{
"name": "AWS.Logger.Core",
"version": "1.5.1",
"package_manager": "nuget",
"path": "Bolo.JobEngine.sln",
"licenses": [
"https://aws.amazon.com/apache-2-0/"
]
}
]
}-
Trigger a new pipeline on the default branch
-
Ping
http://<HOST>/<PROJECT_FULL_PATH>/-/dependencies.json
The following error will be triggered:
NoMethodError at /root/test-dependency/-/dependencies.json
undefined method `fetch' for nil:NilClass
next ancestor unless ancestor.fetch(:iid, false) # `block (2 levels) in augment_ancestors!': undefined method `fetch' for nil:NilClass The expected behaviour is the following:
{"report":{"status":"ok","job_path":"/root/test-dependency/builds/514","generated_at":"2023-06-08T23:03:03.784Z"},"dependencies":[{"name":"Microsoft.AspNetCore.Http","packager":"C# (Nuget)","version":"2.1.1","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":[{"name":"RSA.Bolo.Logging","version":"1.0.16"},{"name":"NLog.Web.AspNetCore","version":"4.9.0"}]},"vulnerabilities":[{"name":"Cookie parsing failure","severity":"high","id":760,"url":"http://gdk.test:3000/root/test-dependency/-/security/vulnerabilities/760"}],"licenses":[]},{"name":"NLog.Web.AspNetCore","packager":"C# (Nuget)","version":"4.9.0","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":null},"vulnerabilities":[],"licenses":[]},{"name":"RSA.Bolo.Logging","packager":"C# (Nuget)","version":"1.0.16","location":{"blob_path":"/root/test-dependency/-/blob/e9b591c01af1f398dd66d25e2149da0070a5d24b/BoloServices.JobFramework.Workflow.Activities/packages.lock.json","path":"BoloServices.JobFramework.Workflow.Activities/packages.lock.json","top_level":false,"ancestors":null},"vulnerabilities":[],"licenses":[]}]}Example Project
See internal note. Example project: https://gitlab.com/dharris/414559/
What is the current bug behavior?
Accessing the /-/dependencies/ page or dependencies.json from an impacted project gives a 500 error with exception NoMethodError: undefined method 'fetch' for nil:NilClass.
What is the expected correct behavior?
The user should be able to access project Dependencies information.
Relevant logs and/or screenshots
Kibana: https://log.gprd.gitlab.net/goto/ef6d2100-1126-11ee-a017-0d32180b1390
Stack trace:
ee/lib/gitlab/ci/reports/dependency_list/report.rb:72:in `block (2 levels) in augment_ancestors!',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:71:in `map!',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:71:in `block in augment_ancestors!',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:61:in `each_value',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:61:in `augment_ancestors!',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:19:in `dependencies',
ee/lib/gitlab/ci/reports/dependency_list/report.rb:38:in `apply_license',
ee/lib/gitlab/ci/parsers/security/dependency_list.rb:59:in `block in apply_licenses!',
ee/lib/gitlab/ci/parsers/security/dependency_list.rb:58:in `each',
ee/lib/gitlab/ci/parsers/security/dependency_list.rb:58:in `apply_licenses!',
ee/app/models/ee/ci/pipeline.rb:138:in `block in dependency_list_report',
ee/app/models/ee/ci/pipeline.rb:128:in `dependency_list_report',
ee/app/services/security/dependency_list_service.rb:34:in `init_collection',
ee/app/services/security/dependency_list_service.rb:23:in `execute',
ee/lib/api/dependencies.rb:16:in `dependencies_by',
ee/lib/api/dependencies.rb:48:in `block (2 levels) in <class:Dependencies>',
ee/lib/gitlab/middleware/ip_restrictor.rb:14:in `block in call',
ee/lib/gitlab/ip_address_state.rb:10:in `with',
ee/lib/gitlab/middleware/ip_restrictor.rb:13:in `call',
lib/api/api_guard.rb:219:in `call',
ee/lib/omni_auth/strategies/group_saml.rb:41:in `other_phase',
lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call',
lib/gitlab/middleware/memory_report.rb:13:in `call',
lib/gitlab/middleware/speedscope.rb:13:in `call',
lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call',
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call',
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call',
lib/gitlab/metrics/web_transaction.rb:46:in `run',
lib/gitlab/metrics/rack_middleware.rb:16:in `call',
lib/gitlab/jira/middleware.rb:19:in `call',
lib/gitlab/middleware/go.rb:20:in `call',
lib/gitlab/etag_caching/middleware.rb:21:in `call',
lib/gitlab/middleware/query_analyzer.rb:11:in `block in call',
lib/gitlab/database/query_analyzer.rb:37:in `within',
lib/gitlab/middleware/query_analyzer.rb:11:in `call',
lib/gitlab/middleware/multipart.rb:173:in `call',
lib/gitlab/middleware/read_only/controller.rb:50:in `call',
lib/gitlab/middleware/read_only.rb:18:in `call',
lib/gitlab/middleware/same_site_cookies.rb:27:in `call',
lib/gitlab/middleware/basic_health_check.rb:25:in `call',
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call',
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call',
lib/gitlab/middleware/request_context.rb:15:in `call',
lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call',
config/initializers/fix_local_cache_middleware.rb:11:in `call',
lib/gitlab/middleware/compressed_json.rb:45:in `call',
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call',
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call',
lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call',
lib/gitlab/middleware/release_env.rb:13:in `call'Output of checks
Results of GitLab environment info
GitLab.com version: GitLab Enterprise Edition 16.1.0-pre 9e2ea67b
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)(we will only investigate if the tests are passing)
Possible fixes
The issue happens because dependency_files[].dependencies[].dependency_path (link to security report schema) includes an iid which hasn't been defined. In the above example, the undefined iid is 52.
Those are some suggestions as possible fixes:
- Update the analyser(s) in order to only allow existing
iid's as part ofdependency_path - Update parser, formatter and/or report so we can either consider this report as invalid or to just ignore the undefined dependency.
Without looking at the bigger picture the following snippet would be a way of ignoring the undefined dependency:
diff --git a/ee/lib/gitlab/ci/reports/dependency_list/report.rb b/ee/lib/gitlab/ci/reports/dependency_list/report.rb
index 0cc769a5962e..fede268df9e2 100644
--- a/ee/lib/gitlab/ci/reports/dependency_list/report.rb
+++ b/ee/lib/gitlab/ci/reports/dependency_list/report.rb
@@ -72,7 +72,7 @@ def augment_ancestors!
next ancestor unless ancestor.fetch(:iid, false)
dependencies_by_iid[ancestor[:iid]]
- end
+ end.compact!
end
end
end
(END)- All the above
Implementation plan
Edited by Duncan