Skip to content
Snippets Groups Projects

Remove the need to download knapsack report from gcs bucket

Merged Remove the need to download knapsack report from gcs bucket
andrey-knapsack-remove-gcs-dep into master
Merged Andrejs Cunskis requested to merge andrey-knapsack-remove-gcs-dep into master

What does this MR do and why?

Refactor knapsack setup to use runtime data stored in repository for knapsack report creation.

This change removes following:

  • Separate jobs that download knapsack report before tests get executed
  • Specific selective report creation as it's not needed, main report has all the runtime data necessary for knapsack (knapsack report doesn't have to contain exactly the same tests that would be executed, it just needs to contain the necessary specs but it can contain all the extra specs as well)
  • Uploading knapsack report itself

Added features:

  • Create "knapsack" report before test run based on example runtimes and take in to account which examples are skipped dynamically to correctly resemble runtime (if a test is quarantined, knapsack needs to have the information it will take miliseconds to run and distribute tests accordingly)
  • Upload updated runtime data for each example on every master execution

Closes Simplify knapsack setup (gitlab-org/quality/analytics/team#5 - closed)

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Andrejs Cunskis

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Andrejs Cunskis changed milestone to %17.8

    changed milestone to %17.8

  • Andrejs Cunskis added groupdevelopment analytics maintenancepipelines typemaintenance labels

    added groupdevelopment analytics maintenancepipelines typemaintenance labels

  • Andrejs Cunskis assigned to @acunskis

    assigned to @acunskis

  • 🤖 GitLab Bot 🤖 added pipelinetier-3 pipeline:run-e2e-omnibus-once labels

    added pipelinetier-3 pipeline:run-e2e-omnibus-once labels

  • Andrejs Cunskis changed the description

    changed the description

  • A deleted user added QA label

    added QA label

  • Ghost User
    Ghost User @ghost1
    Contributor
    3 Warnings
    :warning: This merge request is quite big (1013 lines changed), please consider splitting it into multiple merge requests.
    :warning: e011ab78: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
    :warning: This merge request added or updated artifact settings in CI YAML files.
    Please set restricted access if artifacts contain sensitive data.

    Pipeline Changes

    This merge request contains changes to the pipeline configuration for the GitLab project.

    Please consider the effect of the changes in this merge request on the following:

    Please consider communicating these changes to the broader team following the communication guideline for pipeline changes

    Artifact settings updated in CI YAML files

    The following files have updated artifact settings:

    • .gitlab/ci/qa-common/main.gitlab-ci.yml

    Please verify if these artifacts contain sensitive data. If so, use the artifacts:access keyword to restrict download access.

    Example of setting restricted access:

    artifacts:
  access: developer

    Reviewer roulette

    Category Reviewer Maintainer
    QA @kkolpakova profile link current availability (UTC+1, 1 hour behind author) @ddavison profile link current availability (UTC-5, 7 hours behind author)
    ~"Pipeline configuration" Reviewer review is optional for ~"Pipeline configuration" @ddavison profile link current availability (UTC-5, 7 hours behind author)

    Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

    Edited by Ghost User
  • Ghost User
    Ghost User @ghost1
    Contributor

    E2E Test Result Summary

    allure-report-publisher generated test report!

    e2e-test-on-cng: :white_check_mark: test report for 6bd93385

    expand test summary 
    +------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Create      | 136    | 0      | 19      | 1     | 155   | ✅     |
| Plan        | 82     | 0      | 8       | 0     | 90    | ✅     |
| Govern      | 82     | 0      | 10      | 0     | 92    | ✅     |
| Package     | 30     | 0      | 14      | 0     | 44    | ✅     |
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Verify      | 51     | 0      | 15      | 0     | 66    | ✅     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Secure      | 2      | 0      | 5       | 0     | 7     | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 434    | 0      | 118     | 1     | 552   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

    e2e-test-on-gdk: :white_check_mark: test report for 6bd93385

    expand test summary 
    +------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Create      | 135    | 0      | 20      | 0     | 155   | ✅     |
| Package     | 25     | 0      | 13      | 0     | 38    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Govern      | 80     | 0      | 12      | 0     | 92    | ✅     |
| Verify      | 50     | 0      | 16      | 0     | 66    | ✅     |
| Plan        | 82     | 0      | 8       | 0     | 90    | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
| Secure      | 4      | 0      | 3       | 0     | 7     | ✅     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 427    | 0      | 119     | 0     | 546   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

    e2e-test-on-omnibus: :white_check_mark: test report for 56574072

    expand test summary 
    +---------------------------------------------------------------------+
|                           suites summary                            |
+----------------+--------+--------+---------+-------+-------+--------+
|                | passed | failed | skipped | flaky | total | result |
+----------------+--------+--------+---------+-------+-------+--------+
| Package        | 32     | 0      | 13      | 0     | 45    | ✅     |
| Create         | 563    | 0      | 77      | 2     | 640   | ✅     |
| Manage         | 25     | 0      | 20      | 0     | 45    | ✅     |
| Plan           | 83     | 0      | 8       | 0     | 91    | ✅     |
| Data Stores    | 36     | 0      | 21      | 0     | 57    | ✅     |
| Verify         | 49     | 0      | 16      | 1     | 65    | ✅     |
| Systems        | 6      | 0      | 1       | 0     | 7     | ✅     |
| Configure      | 1      | 0      | 3       | 0     | 4     | ✅     |
| Monitor        | 12     | 0      | 13      | 0     | 25    | ✅     |
| GitLab Metrics | 2      | 0      | 1       | 0     | 3     | ✅     |
| Release        | 5      | 0      | 1       | 0     | 6     | ✅     |
| Govern         | 110    | 0      | 9       | 1     | 119   | ✅     |
| Secure         | 5      | 0      | 3       | 0     | 8     | ✅     |
| Fulfillment    | 4      | 0      | 7       | 2     | 11    | ✅     |
| Ai-powered     | 1      | 0      | 2       | 0     | 3     | ✅     |
| Analytics      | 3      | 0      | 0       | 0     | 3     | ✅     |
| Growth         | 0      | 0      | 2       | 0     | 2     | ➖     |
| ModelOps       | 0      | 0      | 1       | 0     | 1     | ➖     |
+----------------+--------+--------+---------+-------+-------+--------+
| Total          | 937    | 0      | 198     | 6     | 1135  | ✅     |
+----------------+--------+--------+---------+-------+-------+--------+
    Edited by Ghost User
  • Andrejs Cunskis added 3 commits

    added 3 commits

    • 7ae8e6af - Remove the need to download knapsack report from gcs bucket
    • a084a5e3 - Store separate spec runtime data
    • 1cafd3bc - Dynamically create knapsack report before execution

    Compare with previous version

  • Andrejs Cunskis added 2 commits

    added 2 commits

    • 7310f607 - Improve formatter configuration for example data fetch
    • 4bc85a2b - Temp save local knapsack report

    Compare with previous version

  • Andrejs Cunskis added 128 commits

    added 128 commits

    • 4bc85a2b...1e6c0f0d - 121 commits from branch andrey-specific-example-data
    • eea497f7 - Remove the need to download knapsack report from gcs bucket
    • ccb62e3d - Store separate spec runtime data
    • 92406885 - Dynamically create knapsack report before execution
    • 694af571 - Improve formatter configuration for example data fetch
    • 9d5b569b - Temp save local knapsack report
    • 472f5652 - Normalize created local report
    • 8bf11410 - Disable fast quarantine formatter for ExampleData fetcher

    Compare with previous version

    Toggle commit list
  • Andrejs Cunskis added 2 commits

    added 2 commits

    • b2deea76 - Move fetching executable examples to runner class
    • 5cacccf7 - Disable various formatters via dry run setting

    Compare with previous version

  • Andrejs Cunskis added 1 commit

    added 1 commit

    • b52d1528 - Disable various formatters via dry run setting

    Compare with previous version

  • Andrejs Cunskis changed the description

    changed the description

  • Andrejs Cunskis added 149 commits

    added 149 commits

    • b52d1528...619f041b - 143 commits from branch andrey-specific-example-data
    • df6688fb - Remove the need to download knapsack report from gcs bucket
    • 7f181aeb - Store separate spec runtime data
    • 447670e8 - Dynamically create knapsack report before execution
    • 569a5599 - Temp save local knapsack report
    • 56cb6ff3 - Normalize created local report
    • c6881ac5 - Move fetching executable examples to runner class

    Compare with previous version

    Toggle commit list
  • Andrejs Cunskis added 1 commit

    added 1 commit

    • 306cf14e - Build report with runtime for all tests

    Compare with previous version

  • Andrejs Cunskis added 1 commit

    added 1 commit

    • bf266d44 - Build report with runtime for all tests

    Compare with previous version

  • Andrejs Cunskis marked this merge request as draft

    marked this merge request as draft

  • Andrejs Cunskis added 1 commit

    added 1 commit

    • 2c751899 - Normalize created local report

    Compare with previous version

  • Andrejs Cunskis added 80 commits

    added 80 commits

    • 2c751899...993a053c - 75 commits from branch andrey-specific-example-data
    • fcbcd6ed - Remove the need to download knapsack report from gcs bucket
    • 7c1b0d75 - Store separate spec runtime data
    • 0134a683 - Dynamically create knapsack report before execution
    • e90d49cd - Temp save local knapsack report
    • 5835bb65 - Normalize created local report

    Compare with previous version

    Toggle commit list
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply