Prevent merge on code quality degradation

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Problem to solve

Some teams need or want to fail a pipeline if there are any degradations in Code quality between the source and target branches that require review and approval. This is similar to the existing pattern of Security approvals already in place in Merge Requests in GitLab. Code Quality checks/failures do not come with the regulatory requirements that security scans do.

Original Issue Writeup

Overview

For GitLab Runner we use the Code quality feature but ended up extending it so it fails the pipeline if there are any reports/degradations. This is one improvement we can make for #33747 (closed)

Proposal

If there are any reports or any new degradations from the Code quality report it should prevent from the merge request from being merged. Since turning it on by default would be a breaking change for most users it might be a good idea to make it configurable.

Intended users

Primary User

• Sasha (Software Developer) - who can confirm that common checks are now completed and resolve issues early in the development cycle that today happen later. This speeds up the time to resolution for these issues by ensuring the context is fresh.

Users who get secondary benefits

• Rachel (Release Manager) - who wants to ensure that common issues are checked for and resolved before approving and executing on a release.
• Delaney (Development Team Lead) - who can now ensure that coding standards across the team are a little more uniform leading to more readable code.
• Simone (Software Engineer in Test) - who can now trust that code quality didn't decrease and focus on testing at the edges of the new functionality.
• Cameron (Compliance Manager) - who wants to enforce standards in the org around code quality as part of the pipeline and day to day developer workflow.

Further details

The Code Quality findings already have a severity, why don’t we have a CodeQuality-Check rule (or Code-Quality-Check) similar to the Vulnerability-Check rule?

I propose that anything with a severity of major, critical, or blocker trigger a dynamic rule when one is put in place to prevent the code from being merged.

Use Cases

• When Code Quality decreases, teams want a pipeline to fail, so that they can ensure that the code quality and standards the team agreed to are being met.
• When Code Quality standards change, a team wants to be able to update the threshold for failing a pipeline, so that the job failure is not ignored and code quality standards are being kept up.
• We have heard that some users do not want this / would not enable it without the ability to whitelist / ignore some rules on a project by project basis. As such this likely needs to follow or deploy alongside #221237

Proposal

1. Existing jobs and users consuming the existing template will continue to have jobs pass after completion regardless of the outcome
2. Users can customize the thresholds for jobs / turn off jobs that are too noise through existing customization of the codeclimate.yml file.
3. A logical follow-on feature set would be to build a code quality dashboard that enables functionality like enabling/disabling checks though a UI, logging specific issues in specific files to track progress against through GitLab Issues and visualizing code quality over time.

Locations where this would be implemented

• ee/app/models/concerns/approval_rule_like.rb
• ee/app/assets/javascripts/approvals/components/rule_name.vue
• ee/app/assets/javascripts/approvals/mount_project_settings.js
• ee/spec/frontend/approvals/components/rule_name_spec.js
• ee/spec/frontend/approvals/components/security_configuration/unconfigured_security_rules_spec.js
• ee/app/assets/javascripts/approvals/components/security_configuration/unconfigured_security_rules.vue
• ee/app/assets/javascripts/approvals/constants.js

Permissions and Security

Documentation

• Update existing documentation about how to modify the job to fail on code quality degradation.

Availability & Testing

What does success look like, and how can we measure that?

• This feature is being implemented as part of a ~dogfooding effort so success is that internal GitLab teams start using Code Quality in their own pipelines.
• Externally we expect to see no decrease in the number of customers using the code quality job in their pipelines.

What is the type of buyer?

This feature really starts to be valuable when a team or entire engineering organization is trying to enforce standards in coding OR has a regulatory requirement to show a level of testing through static checks so the likely buyer is a Manager or above putting this in the ~"GitLab Starter" tier.

Is this a cross-stage feature?

No.

Links / references