List pipeline reports from self and child pipelines
What does this MR do?
Related to #215725 (closed)
The goal of this MR is to allow MR widgets to display reports generated by child pipelines when parent-child pipelines hierarchy is used. Today the MR widget only displays reports generated inside the MergeRequest#actual_head_pipeline
but if the actual_head_pipeline
has child pipelines that also create reports, these are not visible in the MR widgets.
This MR changes the method that exposes report artifacts for a given pipeline to also return report artifacts for any given child pipelines.
The purpose of using parent-child pipelines is to break down a large and complex pipeline into small manageable child pipelines. These child pipelines are not standalone and their purpose is to contribute to the outcomes of their parent pipeline.
According to this definition it makes sense that a parent pipeline also lists reports from its own child pipelines.
Documentation will be added prior to enabling the feature flag #273200
Screenshots (strongly suggested)
Considering having the following .gitlab-ci.yml
:
build:
stage: build
script:
- echo "building..."
- echo "line_of_code 1000" > metrics.txt
artifacts:
reports:
metrics: metrics.txt # this metrics report will be shown in the MR widget
# as it's being defined in the parent pipeline (existing behavior today)
trigger-tests:
stage: test
trigger:
include: child.yml
and child.yml
being:
rspec:
script:
- echo "Running tests..."
- cat junit-example.xml > rspec.xml # simulate a junit XML report
artifacts:
paths: [rspec.xml]
reports:
junit: rspec.xml
We used a stub report junit-example.xml
for this example rather than generating it
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="4" skipped="0" failures="2" errors="0" time="0.011289" timestamp="2018-07-17T10:48:13+00:00" hostname="runner-400e3f62-project-15-concurrent-0">
<properties>
<property name="seed" value="404"/>
</properties>
<testcase classname="spec.test_spec" name="Test#sum when a is 1 and b is 2 returns summary" file="./spec/test_spec.rb" time="0.009292"><failure message="
expected: 3
got: -1
(compared using ==)
" type="RSpec::Expectations::ExpectationNotMetError">Failure/Error: is_expected.to eq(3)
expected: 3
got: -1
(compared using ==)
./spec/test_spec.rb:12:in `block (4 levels) in <top (required)>'</failure></testcase>
<testcase classname="spec.test_spec" name="Test#sum when a is 100 and b is 200 returns summary" file="./spec/test_spec.rb" time="0.000180"><failure message="
expected: 300
got: -100
(compared using ==)
" type="RSpec::Expectations::ExpectationNotMetError">Failure/Error: is_expected.to eq(300)
expected: 300
got: -100
(compared using ==)
./spec/test_spec.rb:21:in `block (4 levels) in <top (required)>'</failure></testcase>
<testcase classname="spec.test_spec" name="Test#subtract when a is 1 and b is 2 raises an error" file="./spec/test_spec.rb" time="0.000748"></testcase>
<testcase classname="spec.test_spec" name="Test#subtract when a is 2 and b is 1 returns correct result" file="./spec/test_spec.rb" time="0.000064"></testcase>
</testsuite>
include_child_pipeline_jobs_in_reports
enabled:
With the new feature flag We show reports generate by jobs in the parent pipeline as well any child pipelines in the hierarchy as long as the child pipeline was triggered using strategy:depend
.
If the child pipeline wasn't triggered using strategy:depend
then the reports won't be visible from the parent pipeline, hence not displayed.
include_child_pipeline_jobs_in_reports
disabled (existing behavior):
With the new feature flag We show only the metrics report because it's defined in the parent pipeline, but not the JUnit report from the dependent child pipeline.
Query plans
pipeline.latest_report_builds
When FF include_child_pipeline_jobs_in_reports
is enabled it queries the build options which touches the feature flag ci_build_metadata_config
that today is disabled by default (including Gitlab.com)
include_child_pipeline_jobs_in_reports: false
):
before (FF SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build'
AND "ci_builds"."commit_id" = 208831026 AND ("ci_builds"."retried" = false OR "ci_builds"."retried" IS NULL)
AND (EXISTS (
SELECT 1 FROM "ci_job_artifacts" WHERE (ci_builds.id = ci_job_artifacts.job_id)
AND "ci_job_artifacts"."file_type" IN (4, 5, 6, 7, 8, 9, 10, 101, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)))
first run: https://explain.depesz.com/s/bh19 second run: https://explain.depesz.com/s/VBlf
include_child_pipeline_jobs_in_reports: true
; FF ci_build_metadata_config: false
):
after (FF SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build'
AND ("ci_builds"."retried" = false OR "ci_builds"."retried" IS NULL)
AND "ci_builds"."commit_id" IN (
WITH RECURSIVE "base_and_descendants" AS (
(SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 208831026)
UNION
(SELECT "ci_pipelines".* FROM "ci_pipelines", "base_and_descendants", "ci_sources_pipelines", "ci_builds"
WHERE "ci_sources_pipelines"."pipeline_id" = "ci_pipelines"."id"
AND "ci_sources_pipelines"."source_pipeline_id" = "base_and_descendants"."id"
AND "ci_sources_pipelines"."source_project_id" = "ci_sources_pipelines"."project_id"
AND "ci_sources_pipelines"."source_job_id" = "ci_builds"."id"
AND "ci_builds"."options" LIKE '%strategy: depend%')
) SELECT id FROM "base_and_descendants" AS "ci_pipelines")
AND (EXISTS (
SELECT 1 FROM "ci_job_artifacts" WHERE (ci_builds.id = ci_job_artifacts.job_id)
AND "ci_job_artifacts"."file_type" IN (4, 5, 6, 7, 8, 9, 10, 101, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)))
first run: https://explain.depesz.com/s/2dO5 second run: https://explain.depesz.com/s/Zi4A
include_child_pipeline_jobs_in_reports: true
; FF ci_build_metadata_config: true
):
after (FF Potential scenario if ci_build_metadata_config
will be enabled.
SELECT "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" = 'Ci::Build'
AND ("ci_builds"."retried" = false OR "ci_builds"."retried" IS NULL)
AND "ci_builds"."commit_id" IN (
WITH RECURSIVE "base_and_descendants" AS (
(SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = 208831026)
UNION
(SELECT "ci_pipelines".* FROM "ci_pipelines", "base_and_descendants", "ci_sources_pipelines", "ci_builds", "ci_builds_metadata"
WHERE "ci_sources_pipelines"."pipeline_id" = "ci_pipelines"."id"
AND "ci_sources_pipelines"."source_pipeline_id" = "base_and_descendants"."id"
AND "ci_sources_pipelines"."source_project_id" = "ci_sources_pipelines"."project_id"
AND "ci_sources_pipelines"."source_job_id" = "ci_builds"."id"
AND "ci_builds"."id" = "ci_builds_metadata"."build_id"
AND (ci_builds_metadata.config_options->'trigger'->>'strategy' = 'depend'))
) SELECT id FROM "base_and_descendants" AS "ci_pipelines")
AND (EXISTS (
SELECT 1 FROM "ci_job_artifacts" WHERE (ci_builds.id = ci_job_artifacts.job_id)
AND "ci_job_artifacts"."file_type" IN (4, 5, 6, 7, 8, 9, 10, 101, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26)))
first run: https://explain.depesz.com/s/oVf second run: https://explain.depesz.com/s/3pVS
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team