Fail to generate report for dotnet on license-finder project due to asdf plugin curl redirect

Summary

The License Scanning job for the csharp-nuget-dotnetcore failed https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore/-/jobs/1041460022

After some investigation I found that this was due to the third party asdf plugin using a curl command https://github.com/emersonsoares/asdf-dotnet-core/blob/master/bin/install#L5

which was being redirected

>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
} [5 bytes data]
< HTTP/2 302
< cache-control: no-store
< location: /download/dotnet/scripts/v1/dotnet-install.sh
< x-cache: TCP_MISS
< server: Kestrel
< request-context: appId=cid-v1:109cec76-ff6c-4be0-97b5-4f02dbac2451
< x-powered-by: ASP.NET
< x-content-type-options: nosniff
< x-azure-ref: 0VIUvYAAAAAC5fC83hO+LQoZB5vEay5PcTE9OMjFFREdFMDIxNQBlMWRmMDcwYS1hZTQ0LTRjMGItYTU0Yi1jNDkzODA0ZTRkOWY=
< date: Fri, 19 Feb 2021 09:30:59 GMT
< content-length: 0
<
{ [0 bytes data]
* Connection #0 to host dotnet.microsoft.com left intact

A 0 byte file was being created and executed. As far as the script was concerned, there were no errors. However this meant that dotnet was not installed, we couldn't build, the license scanning job returns an empty report.

Steps to reproduce

Run license-scanning job for https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore

Or run # asdf plugin-test dotnet-core https://github.com/emersonsoares/asdf-dotnet-core.git 'dotnet --version' to test the plugin

Example Project

https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore

What is the current bug behavior?

License Scanning returns empty report as third party plugin fails to download dotnet installer

What is the expected correct behavior?

License Scanning returns correct report and third party plugin has downloaded dotnet installer

Relevant logs and/or screenshots

Output of checks

GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

I raised issue https://github.com/emersonsoares/asdf-dotnet-core/issues/26 and MR https://github.com/emersonsoares/asdf-dotnet-core/pull/27 with the third party plugin to add the -L option to curl to follow redirects

This is successful locally

# asdf plugin-test dotnet-core https://github.com/willmeek/asdf-dotnet-core.git --asdf-plugin-gitref add_redirect_to_curl_command  'dotnet --version'
warning: templates not found in /usr/share/git-core/templates
Updating dotnet-core...
From https://github.com/willmeek/asdf-dotnet-core
 * [new branch]      add_redirect_to_curl_command -> add_redirect_to_curl_command
Switched to branch 'add_redirect_to_curl_command'
Downloading the CLI installer
Installing the CLI requested version 5.0.103. Please wait, installation may take a few minutes.
dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.

dotnet-install: Downloading primary link https://dotnetcli.azureedge.net/dotnet/Sdk/5.0.103/dotnet-sdk-5.0.103-linux-x64.tar.gz
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/5.0.103/dotnet-sdk-5.0.103-linux-x64.tar.gz
dotnet-install: Binaries of dotnet can be found in /tmp/asdf.RbXy/installs/dotnet-core/5.0.103
dotnet-install: Note that the script does not resolve dependencies during installation.
dotnet-install: To check the list of dependencies, go to https://docs.microsoft.com/dotnet/core/install, select your operating system and check the "Dependencies" section.
dotnet-install: Installation finished successfully.
The CLI has been installed.
5.0.103

In future, in order for us to fail fast, testing (eg. plugin-test)/error checking could be introduced at https://gitlab.com/gitlab-org/security-products/analyzers/license-finder/-/blob/main/config/projects/dotnet.rb

Potential workaround

See https://gitlab.com/gitlab-org/security-products/tests/csharp-nuget-dotnetcore/-/blob/fork_asdf_dotnet/.gitlab-ci.yml where we use a fork of the plugin

Implementation plan

  • create a fork of https://github.com/emersonsoares/asdf-dotnet-core where redirections are handled properly
  • update and release license-finder
    • change config/software/asdf.rb to use the fork of asdf-dotnet-core
    • drop a code comment about reverting to the official asdf-dotnet-core when possible
    • bump the license-management Ruby gem
    • update the changelog
    • release a new version
Edited by Fabien Catteau