Skip to content

Refactor database testing output structure

Simon Tomlinson requested to merge refactor-database-testing-directory-structure into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Gitlab side of Support multiple versions of input files (gitlab-org/database-team/gitlab-com-database-testing!86 - merged)

Moves each migration to its own directory, and begin versioning the directory structure.

We're changing the directory structure from

migration_testing/
    up/
        v1_name1_query-details.json
        v1_name1.log
        v1_name1_transaction-duration.json
        v2_name2_query_details.json
        v2_name2.log
        v2_name2_transaction-duration.json

to

migration_testing/
    up/
        name1/
            query-details.json
            migration.log
            transaction-duration.json
        name2/
            query-details.json
            migration.log
            transaction-duration.json

This is a step towards testing more than just normal migrations, since it lets us change the notifier code to accept an arbitrary directory and parse consistently-named files within.

Additionally, this MR adds a version file that indicates the current folder structure being produced. This will make it much easier to change the directory structure in the future.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

Running this against a migration locally, I get the following, expected directory structure:

migration-testing/
└── up
    ├── BackfillAllProjectNamespaces
    │   ├── migration.log
    │   ├── query-details.json
    │   └── transaction-duration.json
    ├── metadata.json
    └── migration-stats.json

2 directories, 5 files

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Set up your local environment so that you have at least 1 migration to run. You might have some pending, or you could rollback the latest one with bin/rails db:rollback:main STEP=1

Run migration testing locally: rake gitlab:db:migration_testing:up

See the files in tmp/migration-testing

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Simon Tomlinson

Merge request reports