Skip to content

Introduce the explicit definition for production environments [RUN ALL RSPEC] [RUN AS-IF-FOSS]

Shinya Maeda requested to merge environment-tier into master

What does this MR do?

This MR introduces the explicit definition of environment types (tiers). One of production, staging, testing, development or other can be assigned to each environment. GitLab has some features that particularly target production environments. Until today, we've been performing expensive text/glob matching against environment name. This typically doesn't scale especially in medium/large size projects.

Also, we have inconsistencies in the code base that definition of production differs per feature domain e.g. Jira integration and VSA has different production definition. This should be fixed by having SSOT definition.

FYI, environment_type is already occupied for environment grouping. The existing feature cannot be used for #291746 (closed) and #300741 (closed) (The primary goals) because it's still user-defined value, which has not been normalized for the system. The terminology "tier" was chosen from the Deployment environment. This tier is unrelated to GitLab paid tiers.

In upcoming MRs, we'll allow users to define the environment tier from .gitlab-ci.yml. If the tier is explicitly specified, the value will always override the existing tier.

This feature is disabled by default by environment_tier feature flag.

You can see the full PoC in this MR.

Related #291746 (closed) #300741 (closed)

Database Migration

shinya@shinya-B550-VISION-D:~/workspace/thin-gdk/services/rails/src$ tre bin/rails db:migrate:redo VERSION=20210301150451
INFO: This script is a predefined script in devkitkat.
== 20210301150451 AddTierToEnvironments: reverting ============================
-- remove_column(:environments, :tier)
   -> 0.0038s
== 20210301150451 AddTierToEnvironments: reverted (0.0082s) ===================

== 20210301150451 AddTierToEnvironments: migrating ============================
-- add_column(:environments, :tier, :smallint)
   -> 0.0013s
== 20210301150451 AddTierToEnvironments: migrated (0.0038s) ===================
shinya@shinya-B550-VISION-D:~/workspace/thin-gdk/services/rails/src$ tre bin/rails db:migrate:redo VERSION=20210303091651
INFO: This script is a predefined script in devkitkat.
== 20210303091651 AddIndexToEnvironmentsTier: reverting =======================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:environments, :state, {:name=>"index_environments_on_project_id_and_tier", :algorithm=>:concurrently})
   -> 0.0031s
== 20210303091651 AddIndexToEnvironmentsTier: reverted (0.0035s) ==============

== 20210303091651 AddIndexToEnvironmentsTier: migrating =======================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:environments, [:project_id, :tier], {:where=>"tier IS NOT NULL", :name=>"index_environments_on_project_id_and_tier", :algorithm=>:concurrently})
   -> 0.0023s
== 20210303091651 AddIndexToEnvironmentsTier: migrated (0.0026s) ==============

Query plan with index

In the upcoming MR, we'll execute Environment.for_project(target_projects).for_tier(environment_tier) for searching environments that fall into a particular project and tier.

Tested on database-lab:

Preparation

Targeting https://gitlab.com/gitlab-org/gitlab/-/environments/1178942.

exec ALTER TABLE "environments" ADD "tier" smallint
exec CREATE  INDEX CONCURRENTLY "index_environments_on_project_id_and_tier" ON "environments"  ("project_id", "tier") WHERE tier IS NOT NULL
exec UPDATE environments (tier) VALUES (0) WHERE id = 1178942;

Query

SELECT * FROM environments WHERE projects IN (278964) AND tier = 0 AND tier IS NOT NULL

Timing

Time: 1.407 ms
  - planning: 1.318 ms
  - execution: 0.089 ms
    - I/O read: N/A
    - I/O write: N/A

Shared buffers:
  - hits: 5 (~40.00 KiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Plan

Index Scan using index_environments_on_project_id_and_tier on public.environments  (cost=0.14..145.34 rows=110 width=159) (actual time=0.063..0.063 rows=1 loops=1)
   Index Cond: ((environments.project_id = 278964) AND (environments.tier = 0))
   Buffers: shared hit=5

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Shinya Maeda

Merge request reports