Skip to content

Draft: Add gitlab schema validator

Patrick Bair requested to merge pb-gitlab-schema-validator into master

What does this MR do and why?

Finish the implementation for a new query analyzer from !71540 (closed). If the allowed schemas are configured in database.yml, it checks that a certain connection does not access tables outside the specified schemas.

The analyzer only applies to queries and DML. DDL is always allowed on a read/write connection, since we intend to keep the schema in sync.

How to set up and validate locally

  1. Add the following lines under the main database configuration in database.yml:
    gitlab_primary_schemas: [gitlab_main, gitlab_shared]
    gitlab_replica_schemas: [gitlab_main, gitlab_shared]
  2. Add the following lines under the ci database configuration in database.yml:
    gitlab_primary_schemas: [gitlab_ci, gitlab_shared]
    gitlab_replica_schemas: [gitlab_ci, gitlab_shared]
  3. In a rails console, test querying the database using both connections:
    Gitlab::Database::QueryAnalyzer.instance.within { ActiveRecord::Base.connection.select_value('select count(*) from projects')  } # => ok
    Gitlab::Database::QueryAnalyzer.instance.within { Ci::ApplicationRecord.connection.select_value('select count(*) from ci_builds')  } # => ok
    Gitlab::Database::QueryAnalyzer.instance.within { ActiveRecord::Base.connection.select_value('select count(*) from ci_builds')  } # => error
    Gitlab::Database::QueryAnalyzer.instance.within { Ci::ApplicationRecord.connection.select_value('select count(*) from projects')  } # => error
  4. Test DML using both connections:
    Gitlab::Database::QueryAnalyzer.instance.within { ActiveRecord::Base.connection.execute('update projects set updated_at = now()') } # => ok
    Gitlab::Database::QueryAnalyzer.instance.within { Ci::ApplicationRecord.connection.execute('update ci_builds set updated_at = now()') } # => ok
    Gitlab::Database::QueryAnalyzer.instance.within { ActiveRecord::Base.connection.execute('update ci_builds set updated_at = now()') } # => error
    Gitlab::Database::QueryAnalyzer.instance.within { Ci::ApplicationRecord.connection.execute('update projects set updated_at = now()') } # => error
  5. Test DDL using both connections:
    Gitlab::Database::QueryAnalyzer.instance.within { ActiveRecord::Base.connection.add_column(:ci_builds, :foobar, :integer) } # => ok
    Gitlab::Database::QueryAnalyzer.instance.within { Ci::ApplicationRecord.connection.add_column(:projects, :foobar, :integer) } # => ok

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Patrick Bair

Merge request reports