Skip to content

Add CreateArtifactService to persist code_coverage report

Max Orefice requested to merge mo-add-pipeline-artifact-service into master

Part of #211410 (closed)

Step 3 to make code coverage available for everyone.

This feature is behind a feature flag (coverage_report_view). The full feature has been developed on this POC branch.

The team decided to implement this feature with this approach according to this proposal validated by @ayufan.

What does this MR do?

This MR creates a new service which persists generated code_coverage report once a pipeline has been completed.

It includes the following items:

  • Adds Ci::Pipelines::CreateArtifactWorker
  • Adds Ci::Pipelines::CreateArtifactService

Why are we doing this?

Today our Code Coverage feature does not scale properly for large projects containing a lot of code to analyze.

Processing data in memory and parsing big chunk of data have some ~performance implication as explained on this issue.

Here a diagram of tomorrow's architecture:

sequenceDiagram
    participant MergeRequest
    participant Pipeline
    participant PipelineArtifact
    MergeRequest->>Pipeline: has_coverage_reports?
    Note right of Pipeline: When pipeline is completed
    loop PipelineArtifactService
        Pipeline->>PipelineArtifact: Persist coverage report with object storage
    end
    Note right of MergeRequest: No more ReactiveCaching
    Pipeline->>MergeRequest: Read file from object storage

How the feature looks like?

You can visualize how this feature is showing up on the UI on our demo project.

Context

This MR represents a vertical slice of functionality for improving the ~performance of our Code coverate feature. This is slice 3/4.

This work will occur in 4 slices:

  1. Create data model - !37969 (merged)
  2. Introduce a new uploader - !38221 (merged)
  3. Process and persist reports when a pipeline is finished -
  4. Fetch report from new abstraction -

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Max Orefice

Merge request reports