Release evidence for report artifacts links to nonexistent evidence

Summary

Release evidence collection of test results, as implemented in #32773 (closed), collects release evidence from jobs with no artifacts:paths, and therefore the report_artifacts.url points to non-existent evidence (returns a 404 page).

Steps to reproduce

Create a release for a project with pipelines that include the predefined security jobs.

Example Project

An example project is included at https://gitlab.com/aarongoldenthal/release-evidence-test/. See release 0.1.0 (the other releases are affected by #225797 (closed)).

What is the current bug behavior?

(What actually happens)

What is the expected correct behavior?

The release evidence report_artifacts.url should point to downloadable content. This could require artifacts:paths to be defined for the jobs, but if that's the case then applying that to the predefined templates needs to be addressed, because it's not clear how to do that with most of the non Docker-in-Docker security templates.

Relevant logs and/or screenshots

Below is a copy of the release artifacts identified in the example project above:

{
    "release": {
        "id": 1607038,
        "name": "0.1.0",
        "project": {
            "id": 19670924,
            "name": "Release Artifacts Test",
            "created_at": "2020-06-30T01:18:49.391Z",
            "description": "Test  collection of release artifacts from GitLab CI templates."
        },
        "tag_name": "0.1.0",
        "created_at": "2020-06-30T01:32:45.259Z",
        "milestones": [],
        "description": "Release and tag created from UI.  No artifact retention defined beyond templates (which includes `code_quality`).",
        "report_artifacts": [
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655104/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655119/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655117/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655107/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655120/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655122/artifacts/download"
            },
            {
                "url": "https://gitlab.com/aarongoldenthal/release-artifacts-test/-/jobs/616655123/artifacts/download"
            }
        ]
    }
}

Output of checks

This bug happens on GitLab.com.

Proposal

  1. Remove inaccessible artifacts from evidence. This can be done by removing artifacts without artifacts:paths in build_artifact_entity.

  2. Update the references in CI templates to reflect the above change. This is tracked in #228648 (closed).

Edited by Ahmed Hemdan