Add project scope to live trace feature flag (!31325) · Merge requests · GitLab.org / GitLab FOSS

Shinya Maeda requested to merge add-project-scope-to-live-trace-feature-flag into master Jul 31, 2019

What does this MR do?

This is requested by infra to evaluate new live trace architecture in our projects (e.g. GitLab CE/EE) to avoid production data loss in the other projects. https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/4667#note_197981470

Manual QA

74] pry(main)> Feature.disable('ci_enable_live_trace')
  Feature::FlipperFeature Load (0.3ms)  SELECT "features".* FROM "features" WHERE "features"."key" = $1  [["key", "ci_enable_live_trace"]]
  Feature::FlipperGate Destroy (3.0ms)  DELETE FROM "feature_gates" WHERE "feature_gates"."feature_key" = $1  [["feature_key", "ci_enable_live_trace"]]
=> true
[75] pry(main)> Feature.enable('ci_enable_live_trace', Project.find(55))
  Project Load (0.2ms)  SELECT  "projects".* FROM "projects" WHERE "projects"."id" = $1 LIMIT $2  [["id", 55], ["LIMIT", 1]]
  Feature::FlipperFeature Load (0.1ms)  SELECT "features".* FROM "features" WHERE "features"."key" = $1  [["key", "ci_enable_live_trace"]]
   (0.1ms)  BEGIN
  Feature::FlipperGate Create (0.2ms)  INSERT INTO "feature_gates" ("feature_key", "key", "value", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"  [["feature_key", "ci_enable_live_trace"], ["key", "actors"], ["value", "Project:55"], ["created_at", "2019-07-31 11:00:13.675084"], ["updated_at", "2019-07-31 11:00:13.675084"]]
   (4.0ms)  COMMIT
=> true
[76] pry(main)> 
[77] pry(main)> project = Project.find(55)
  Project Load (0.2ms)  SELECT  "projects".* FROM "projects" WHERE "projects"."id" = $1 LIMIT $2  [["id", 55], ["LIMIT", 1]]
  Route Load (0.1ms)  SELECT  "routes".* FROM "routes" WHERE "routes"."source_id" = $1 AND "routes"."source_type" = $2 LIMIT $3  [["source_id", 55], ["source_type", "Project"], ["LIMIT", 1]]
=> #<Project id:55 root/pipeline-test>
[78] pry(main)> project.builds.last.trace.erase!
  Ci::Build Load (0.4ms)  SELECT  "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = $1 ORDER BY "ci_builds"."id" DESC LIMIT $2  [["project_id", 55], ["LIMIT", 1]]
  Ci::JobArtifact Load (0.1ms)  SELECT  "ci_job_artifacts".* FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."job_id" = $1 AND "ci_job_artifacts"."file_type" = $2 LIMIT $3  [["job_id", 1210], ["file_type", 3], ["LIMIT", 1]]
   (0.1ms)  SELECT "ci_build_trace_chunks"."build_id", "ci_build_trace_chunks"."chunk_index" FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1 AND "ci_build_trace_chunks"."data_store" = $2  [["build_id", 1210], ["data_store", 1]]
  Ci::BuildTraceChunk Destroy (4.1ms)  DELETE FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1  [["build_id", 1210]]
=> nil
[79] pry(main)> project.builds.last.trace.set('test')
  Ci::Build Load (0.4ms)  SELECT  "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = $1 ORDER BY "ci_builds"."id" DESC LIMIT $2  [["project_id", 55], ["LIMIT", 1]]
  Ci::JobArtifact Load (0.2ms)  SELECT  "ci_job_artifacts".* FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."job_id" = $1 AND "ci_job_artifacts"."file_type" = $2 LIMIT $3  [["job_id", 1210], ["file_type", 3], ["LIMIT", 1]]
  Feature::FlipperGate Load (0.2ms)  SELECT "feature_gates".* FROM "feature_gates" WHERE "feature_gates"."feature_key" = $1  [["feature_key", "ci_enable_live_trace"]]
  Ci::BuildTraceChunk Load (0.1ms)  SELECT  "ci_build_trace_chunks".* FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1 ORDER BY "ci_build_trace_chunks"."chunk_index" DESC LIMIT $2  [["build_id", 1210], ["LIMIT", 1]]
   (0.1ms)  SELECT "features"."key" FROM "features"
  Ci::BuildTraceChunk Load (0.2ms)  SELECT  "ci_build_trace_chunks".* FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1 AND "ci_build_trace_chunks"."chunk_index" = $2 LIMIT $3  [["build_id", 1210], ["chunk_index", 0], ["LIMIT", 1]]
   (0.1ms)  BEGIN
  Ci::BuildTraceChunk Create (0.3ms)  INSERT INTO "ci_build_trace_chunks" ("build_id", "chunk_index", "data_store") VALUES ($1, $2, $3) RETURNING "id"  [["build_id", 1210], ["chunk_index", 0], ["data_store", 1]]
   (4.0ms)  COMMIT
=> nil
[80] pry(main)> Feature.enabled?('ci_enable_live_trace', project)
=> true
[81] pry(main)> 
[82] pry(main)> project = Project.find(54)
  Project Load (0.2ms)  SELECT  "projects".* FROM "projects" WHERE "projects"."id" = $1 LIMIT $2  [["id", 54], ["LIMIT", 1]]
  Route Load (0.1ms)  SELECT  "routes".* FROM "routes" WHERE "routes"."source_id" = $1 AND "routes"."source_type" = $2 LIMIT $3  [["source_id", 54], ["source_type", "Project"], ["LIMIT", 1]]
=> #<Project id:54 root/ten-things-to-know-about-gitlab-ci>
[83] pry(main)> project.builds.last.trace.erase!
  Ci::Build Load (0.5ms)  SELECT  "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = $1 ORDER BY "ci_builds"."id" DESC LIMIT $2  [["project_id", 54], ["LIMIT", 1]]
  Ci::JobArtifact Load (0.2ms)  SELECT  "ci_job_artifacts".* FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."job_id" = $1 AND "ci_job_artifacts"."file_type" = $2 LIMIT $3  [["job_id", 1140], ["file_type", 3], ["LIMIT", 1]]
   (0.1ms)  SELECT "ci_build_trace_chunks"."build_id", "ci_build_trace_chunks"."chunk_index" FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1 AND "ci_build_trace_chunks"."data_store" = $2  [["build_id", 1140], ["data_store", 1]]
  Ci::BuildTraceChunk Destroy (0.2ms)  DELETE FROM "ci_build_trace_chunks" WHERE "ci_build_trace_chunks"."build_id" = $1  [["build_id", 1140]]
=> nil
[84] pry(main)> project.builds.last.trace.set('test')
  Ci::Build Load (0.4ms)  SELECT  "ci_builds".* FROM "ci_builds" WHERE "ci_builds"."type" IN ('Ci::Build') AND "ci_builds"."project_id" = $1 ORDER BY "ci_builds"."id" DESC LIMIT $2  [["project_id", 54], ["LIMIT", 1]]
  Ci::JobArtifact Load (0.1ms)  SELECT  "ci_job_artifacts".* FROM "ci_job_artifacts" WHERE "ci_job_artifacts"."job_id" = $1 AND "ci_job_artifacts"."file_type" = $2 LIMIT $3  [["job_id", 1140], ["file_type", 3], ["LIMIT", 1]]
=> #<File:/home/shinya/workspace/thin-gdk/service/rails/src/builds/2019_07/54/1140.log (closed)>
[85] pry(main)> Feature.enabled?('ci_enable_live_trace', project)
=> false

Does this MR meet the acceptance criteria?

Conformity

Changelog entry for user-facing changes, or community contribution. Check the link for other scenarios.
Documentation created/updated or follow-up review issue created
Code review guidelines
Merge request performance guidelines
Style guides
Database guides
Separation of EE specific content

Performance and testing

Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
Tested in all supported browsers

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

Label as security and @ mention @gitlab-com/gl-security/appsec
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
Security reports checked/validated by a reviewer from the AppSec team

Edited Jul 31, 2019 by Shinya Maeda

Add project scope to live trace feature flag