Skip to content

Add BuildReportResult data model

Max Orefice requested to merge mo-add-build-report-result into master

Part of #211838 (closed)

This feature will be behind a feature flag. The full feature has been developed on this POC branch.

What does this MR do?

This MR creates a new data model which will be used to persist our JUnit results after a build is finished.

It includes the following items:

  • Adds table for ci_build_report_results table & model
  • Adds relationship to Ci::Build model
  • Adds relationship to Project (in case we want to partition our data later on)
  • Creates a new json_validator to better control what data we put in our JSON column

Why are we doing this?

Today our JUnit feature is really slow for big projects containing a lot of tests and requires almost 1min on gitlab.com in order to render the data. We are working toward improving the ~performance of this feature and we decided by starting to persist our counter data which is all the data you can see on the screenshot below 👇

Pipeline___GitLab.org___GitLab___GitLab_2020-05-25_14-51-16

Context

This MR represents a vertical slice of functionality for improving the ~performance of our JUnit feature. This is slice 1/3.

This work will occur in 3 slices:

  1. Create data model -
  2. Populate new data once a build is finished - !33093 (merged)
  3. Expose the new API at the pipeline level - !33431 (merged)

Database Review

-----Click here for supplemental database review materials------

Up migration output: Table creation

$ bin/rails db:migrate
== 20200527151413 CreateCiBuildReportResultsTable: migrating ==================
-- create_table(:ci_build_report_results, {:id=>false})
   -> 0.0070s
== 20200527151413 CreateCiBuildReportResultsTable: migrated (0.0071s) =========

$ bin/rails dbconsole
psql (11.7)
Type "help" for help.

gitlabhq_development=# \d ci_build_report_results
                                 Table "public.ci_build_report_results"
   Column   |  Type  | Collation | Nullable |                          Default
------------+--------+-----------+----------+-----------------------------------------------------------
 build_id   | bigint |           | not null | nextval('ci_build_report_results_build_id_seq'::regclass)
 project_id | bigint |           | not null |
 data       | jsonb  |           | not null | '{}'::jsonb
Indexes:
    "ci_build_report_results_pkey" PRIMARY KEY, btree (build_id)
    "index_ci_build_report_results_on_project_id" btree (project_id)
Foreign-key constraints:
    "fk_rails_056d298d48" FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE
    "fk_rails_16cb1ff064" FOREIGN KEY (build_id) REFERENCES ci_builds(id) ON DELETE CASCADE

Rollback table creation

$ bin/rails db:rollback
== 20200527151413 CreateCiBuildReportResultsTable: reverting ==================
-- drop_table(:ci_build_report_results, {:id=>false})
   -> 0.0016s
== 20200527151413 CreateCiBuildReportResultsTable: reverted (0.0027s) =========

Adding foreign key for build

$ bin/rails db:migrate
== 20200527152116 AddForeignKeyToBuildIdOnBuildReportResults: migrating =======
-- add_foreign_key(:ci_build_report_results, :ci_builds, {:column=>:build_id, :on_delete=>:cascade})
   -> 0.0028s
== 20200527152116 AddForeignKeyToBuildIdOnBuildReportResults: migrated (0.0059s)

Rollback foreign key for build

$ bin/rails db:rollback
== 20200527152116 AddForeignKeyToBuildIdOnBuildReportResults: reverting =======
-- remove_foreign_key(:ci_build_report_results, {:column=>:build_id})
   -> 0.0037s
== 20200527152116 AddForeignKeyToBuildIdOnBuildReportResults: reverted (0.0071s)

Adding foreign key for project

$ bin/rails db:migrate
== 20200527152657 AddForeignKeyToProjectIdOnBuildReportResults: migrating =====
-- add_foreign_key(:ci_build_report_results, :projects, {:column=>:project_id, :on_delete=>:cascade})
   -> 0.0032s
== 20200527152657 AddForeignKeyToProjectIdOnBuildReportResults: migrated (0.0043s)

Rollback foreign key for project

$ bin/rails db:rollback
== 20200527152657 AddForeignKeyToProjectIdOnBuildReportResults: reverting =====
-- remove_foreign_key(:ci_build_report_results, {:column=>:project_id})
   -> 0.0044s
== 20200527152657 AddForeignKeyToProjectIdOnBuildReportResults: reverted (0.0078s)

Backend Review

-----Click here for supplemental backend review materials------

The team decided to implement this feature with this approach according to the discussion we had with @ayufan on this MR.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Max Orefice

Merge request reports