Skip to content

DB testing: Artifact full query log for each migration

Alishan Ladhani requested to merge ali/test-migration into master

What does this MR do?

Artifact the query log for each migration that is run in the DB testing pipeline. All files placed in tmp/migration-testing/ are saved as pipeline artifacts (https://gitlab.com/gitlab-org/database-team/gitlab-com-database-testing/-/blob/master/.gitlab-ci.yml#L65).

Related to gitlab-org/database-team/team-tasks#144 (closed)

Screenshots (strongly suggested)

Test pipeline: !60020 (comment 558179849)

Sample log file: 20210422152437.log
D, [2021-05-10T20:28:02.782467 #30] DEBUG -- :   [1m[35m (37.2ms)[0m  [1m[34mSELECT pg_try_advisory_lock(3005434223539248480)[0m
D, [2021-05-10T20:28:02.783087 #30] DEBUG -- :   ↳ lib/tasks/gitlab/db.rake:240:in `block (5 levels) in <main>'
D, [2021-05-10T20:28:02.857533 #30] DEBUG -- :   [1m[35m (73.6ms)[0m  [1m[34mSELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC /*application:test*/[0m
D, [2021-05-10T20:28:02.858060 #30] DEBUG -- :   ↳ lib/tasks/gitlab/db.rake:240:in `block (5 levels) in <main>'
I, [2021-05-10T20:28:02.866516 #30]  INFO -- : Migrating to TestMigration (20210422152437)
D, [2021-05-10T20:28:02.905645 #30] DEBUG -- :   [1m[35m (37.1ms)[0m  [1m[35mBEGIN /*application:test*/[0m
D, [2021-05-10T20:28:02.906594 #30] DEBUG -- :   ↳ lib/gitlab/database.rb:360:in `block in transaction'
D, [2021-05-10T20:28:02.949813 #30] DEBUG -- :   [1m[35m (42.9ms)[0m  [1m[35mALTER TABLE "user_statuses" ADD "test_column" integer /*application:test*/[0m
D, [2021-05-10T20:28:02.950591 #30] DEBUG -- :   ↳ lib/gitlab/database.rb:360:in `block in transaction'
D, [2021-05-10T20:28:02.988987 #30] DEBUG -- :   [1m[36mprimary::SchemaMigration Create (37.0ms)[0m  [1m[32mINSERT INTO "schema_migrations" ("version") VALUES ('20210422152437') RETURNING "version" /*application:test*/[0m
D, [2021-05-10T20:28:02.990018 #30] DEBUG -- :   ↳ lib/gitlab/database.rb:360:in `block in transaction'
D, [2021-05-10T20:28:03.027956 #30] DEBUG -- :   [1m[35m (37.3ms)[0m  [1m[35mCOMMIT /*application:test*/[0m
D, [2021-05-10T20:28:03.028709 #30] DEBUG -- :   ↳ lib/gitlab/database.rb:360:in `block in transaction'
D, [2021-05-10T20:28:03.067224 #30] DEBUG -- :   [1m[36mActiveRecord::InternalMetadata Load (37.4ms)[0m  [1m[34mSELECT "ar_internal_metadata".* FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = 'environment' LIMIT 1 /*application:test*/[0m
D, [2021-05-10T20:28:03.067898 #30] DEBUG -- :   ↳ lib/tasks/gitlab/db.rake:240:in `block (5 levels) in <main>'
D, [2021-05-10T20:28:03.106513 #30] DEBUG -- :   [1m[35m (37.3ms)[0m  [1m[34mSELECT pg_advisory_unlock(3005434223539248480)[0m
D, [2021-05-10T20:28:03.106944 #30] DEBUG -- :   ↳ lib/tasks/gitlab/db.rake:240:in `block (5 levels) in <main>'
D, [2021-05-10T20:28:03.231510 #30] DEBUG -- :   [1m[35m (124.2ms)[0m  [1m[34mselect pg_database_size(current_database()) /*application:test*/[0m
D, [2021-05-10T20:28:03.232102 #30] DEBUG -- :   ↳ lib/gitlab/database/migrations/observers/total_database_size_change.rb:25:in `get_total_database_size'

Screen_Shot_2021-05-10_at_4.37.40_PM

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Alishan Ladhani

Merge request reports