Skip to content

Add classes representing CodeCoverage ClickHouse tables

What does this MR do and why?

Adds 3 classes needed to parse different files:

  1. CodeCoverage::ClickHouse::CategoryOwnersTable: represents a category_owners table in ClickHouse, and offers functionality to create it and push data to it
  2. CodeCoverage::ClickHouse::CoverageMetricsTable: represents a coverage_metrics table in ClickHouse, and offers functionality to create it and push data to it
  3. CodeCoverage::ClickHouse::Table: base class for the above 2 classes

The classes are needed in the context of this work: !355.

Related to: https://gitlab.com/gitlab-org/quality/feature-readiness/team/-/issues/7

How to set up and validate locally

Although not strictly dependent on anything else, to check the correct functioning of these classes many branches need to be fetched locally:

  1. Fetch these branches in this exact order, stacking each branch on top of each other so that the last branch contains the complete logic:

    1. !364 (merged)
    2. !365 (merged)
    3. !367
    4. this branch
    5. !355
  2. Run these commands to fetch a bunch of files locally, and store them in <project-root>/working_dir/:

    curl -o gitlab.lcov https://gitlab.com/gitlab-org/gitlab/-/jobs/11380416469/artifacts/raw/coverage/lcov/gitlab.lcov
    
    curl -o rspec-11380416038.json https://gitlab.com/gitlab-org/gitlab/-/jobs/11380416038/artifacts/raw/rspec/rspec-11380416038.json
    
    curl -o rspec-11380416056.json https://gitlab.com/gitlab-org/gitlab/-/jobs/11380416056/artifacts/raw/rspec/rspec-11380416056.json
    
    curl https://gitlab-org.gitlab.io/gitlab/crystalball/packed-mapping.json.gz | gzip -d > mapping.json
  3. Use these constants and methods instead of the normal ones in CodeCoverage::PipelineArtifacts:

    RSPEC_REPORTS_PRODUCING_JOBS = [
      'rspec_artifact-collector_unit',
      'rspec_artifact-collector_unit_single-redis',
      'rspec_artifact-collector_ee_unit_single-redis',
      'rspec_artifact-collector_ee_unit'
    ].freeze
    
    def find_coverage_report_path
      path = File.join(working_dir, 'rspec_coverage', 'coverage', 'lcov', 'gitlab.lcov')
      return path if File.exist?(path)
    
      raise "Coverage report not found in: #{path}. Can't collate data to be pushed to ClickHouse without it."
    end
    
    def working_dir
      @working_dir ||= Dir.pwd + '/working_dir/'
    end
  4. Put a breakpoint just after this instruction in exe/test-coverage:

    category_owners_table.create  if ENV['CLICKHOUSE_CREATE_CATEGORY_OWNERS_TABLE']  == 'true'
  5. Run ruby exe/test-coverage

  6. Check that the functionality works as expected

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alberto Bignotti

Merge request reports

Loading