Fail to generate report for dotnet on license-finder project due to asdf plugin curl redirect
Summary
The License Scanning job for the csharp-nuget-dotnetcore
failed
https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore/-/jobs/1041460022
After some investigation I found that this was due to the third party asdf
plugin using a curl command https://github.com/emersonsoares/asdf-dotnet-core/blob/master/bin/install#L5
which was being redirected
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 302
< cache-control: no-store
< location: /download/dotnet/scripts/v1/dotnet-install.sh
< x-cache: TCP_MISS
< server: Kestrel
< request-context: appId=cid-v1:109cec76-ff6c-4be0-97b5-4f02dbac2451
< x-powered-by: ASP.NET
< x-content-type-options: nosniff
< x-azure-ref: 0VIUvYAAAAAC5fC83hO+LQoZB5vEay5PcTE9OMjFFREdFMDIxNQBlMWRmMDcwYS1hZTQ0LTRjMGItYTU0Yi1jNDkzODA0ZTRkOWY=
< date: Fri, 19 Feb 2021 09:30:59 GMT
< content-length: 0
<
{ [0 bytes data]
* Connection #0 to host dotnet.microsoft.com left intact
A 0 byte file was being created and executed. As far as the script was concerned, there were no errors. However this meant that dotnet was not installed, we couldn't build, the license scanning job returns an empty report.
Steps to reproduce
Run license-scanning
job for https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore
Or run # asdf plugin-test dotnet-core https://github.com/emersonsoares/asdf-dotnet-core.git 'dotnet --version'
to test the plugin
Example Project
https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore
What is the current bug behavior?
License Scanning returns empty report as third party plugin fails to download dotnet installer
What is the expected correct behavior?
License Scanning returns correct report and third party plugin has downloaded dotnet installer
Relevant logs and/or screenshots
Output of checks
GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
I raised issue https://github.com/emersonsoares/asdf-dotnet-core/issues/26 and MR https://github.com/emersonsoares/asdf-dotnet-core/pull/27 with the third party plugin to add the -L
option to curl
to follow redirects
This is successful locally
# asdf plugin-test dotnet-core https://github.com/willmeek/asdf-dotnet-core.git --asdf-plugin-gitref add_redirect_to_curl_command 'dotnet --version'
warning: templates not found in /usr/share/git-core/templates
Updating dotnet-core...
From https://github.com/willmeek/asdf-dotnet-core
* [new branch] add_redirect_to_curl_command -> add_redirect_to_curl_command
Switched to branch 'add_redirect_to_curl_command'
Downloading the CLI installer
Installing the CLI requested version 5.0.103. Please wait, installation may take a few minutes.
dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.
dotnet-install: Downloading primary link https://dotnetcli.azureedge.net/dotnet/Sdk/5.0.103/dotnet-sdk-5.0.103-linux-x64.tar.gz
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/5.0.103/dotnet-sdk-5.0.103-linux-x64.tar.gz
dotnet-install: Binaries of dotnet can be found in /tmp/asdf.RbXy/installs/dotnet-core/5.0.103
dotnet-install: Note that the script does not resolve dependencies during installation.
dotnet-install: To check the list of dependencies, go to https://docs.microsoft.com/dotnet/core/install, select your operating system and check the "Dependencies" section.
dotnet-install: Installation finished successfully.
The CLI has been installed.
5.0.103
In future, in order for us to fail fast, testing (eg. plugin-test
)/error checking could be introduced at https://gitlab.com/gitlab-org/security-products/analyzers/license-finder/-/blob/main/config/projects/dotnet.rb
Potential workaround
See https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore/-/blob/fork_asdf_dotnet/.gitlab-ci.yml where we use a fork of the plugin
Implementation plan
-
create a fork of https://github.com/emersonsoares/asdf-dotnet-core where redirections are handled properly -
update and release license-finder - change
config/software/asdf.rb
to use the fork ofasdf-dotnet-core
- drop a code comment about reverting to the official
asdf-dotnet-core
when possible - bump the
license-management
Ruby gem - update the changelog
- release a new version
- change