This is the second migration to validate the foreign key synchronously

See https://docs.gitlab.com/ee/development/database/add_foreign_key_to_existing_column.html#add-a-migration-to-validate-the-fk-synchronously

Changelog: added

When we add CI steps to the CI Catalog, we'll need to store the entire
`spec` header section in the table. This commit is the first step
towards migrating from the `inputs` field to a new `spec` field that
will support the needs of both steps and components.

Changelog: added
Issue: #443662

Changelog: added

Changelog: other

Changelog: fixed

Index: index_environments_for_name_search_within_folder

This index was used for a query that was changed in
!146154

- finished as per !145041 (comment 1801378958)
- followed https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#finalize-a-batched-background-migration

Changelog: other

gitlab-org/gitlab!145881 introduced
new partitioned table `p_catalog_resource_component_usages`, but it's
position in `structure.sql` is not the one it should have when
partitions exists. This updates the file to move the table definition to
the right place.

gitlab-org/gitlab!145881 (comment 1818346331)

[skip-db-structure-check]

Changelog: removed

Adds a new partitioned table p_catalog_resource_component_usages.
This table will track when a CI component is included in a project's
pipeline. Only unique records of component-used_by_project-used_date
are inserted.

This data will be used to display component usage popularity
aggregated by component or catalog_resource per 30 day rolling window.

This table is partitioned by month. For now, the
table will drop partitions with data older than 12 months. This
duration will likely be reduced in a follow up issue.

Changelog: added

gitlab-org/gitlab!144882 adds couple
of indexes, but at the wrong position in `structure.sql`.

This restores the expected order (alphabetical by index name).

See also
gitlab-org/gitlab!145294 (comment 1805667098).

[skip-db-structure-check]

- Added user_id, custom_role_id column to member_approvals
- Removed not null constraint from member_id and old_access_level
- Changed unique index
- Added loose foreign key for user_id
- Added index for new columns

Changelog: changed
EE: true

Changelog: added
EE: true

This enables users to run `FILTER=application_settings bin/rake
db:seed_fu` as many times as they want, and allows retries to happen
in CI if needed.

This also reduces the number of updates to the application settings.
Each `update!` causes a validation to occur. Each URL will be
validated by resolving its DNS address, which could be slow or time
out.

Relates to
gitlab-development-kit#2062

This is the first step of partitioning the merge_request_diff_commits
table by int range.

Epic: &11271

Changelog: added

For use in !145704

Changelog: added

Changelog: changed

Changelog: changed

Add cell local schema to gitlab_subscription_histoires table

Change: other

Expiring worker refactor done

Niko Belokolodov authored 1 year ago and Dylan Griffith committed 1 year ago

Backfill RawUsageData records with Default organization

Changelog: other

Michael Becker authored 1 year ago and Adam Hegyi committed 1 year ago

We want to drop the `vulnerability_occurrence_pipelines` table. In
order to do this, we still need the first and latest pipeline ID
stored somewhere to support the existing feature-set

The columns and asynchronous index creation were added in a
[previous MR][0]

This change adds the synchronous index migration to complete the index
creation [process][1]

Follow-up Work
---------------------------

We will need to add loose foreign keys and model association. Both of
these are blocked on this index creation

That work is tracked in the following tasks:

- #443283
- #443284

---

Related to #422382
Resolves #443282

[0]:!144697
[1]:https://docs.gitlab.com/ee/development/database/adding_database_indexes.html#create-indexes-asynchronously

Changelog: added

`bin/rake db:seed_fu` previously would fail if the admin user already
existed. In case someone needs to re-run that command to re-seed
their database, it would be helpful if this fixture were idempotent
and gracefully skip the fixture if the admin already exists.

Relates to
gitlab-development-kit#2062

When `FILTER=merge_requests bin/rake db:seed_fu` was run, previously
merge request diffs were created with an inline Sidekiq job. However,
since Seed Fu wraps the fixture file in one big transaction, this can
lead to idle in transaction timeouts if the creation of merge requests
takes too long.

To avoid this issue, queue the `NewMergeRequestWorker` jobs
in a real Sidekiq job to be run later. This will speed up
the seeding and avoid idle in transaction timeouts.

Relates to
gitlab-development-kit#2062

This MR backfills the semver columns of existing
catalog versions that don't yet have their semver
values populated. The semver is based on the existing
tag values. Partial version numbers (e.g. `1` or `2.1`)
and extended version numbers (e.g. `1.2.3.4`) are normalized
to follow the conventional semantic versioning format.
If the tag value is neither a valid nor partial/extended
semantic version number, the semver values remain nil.

Changelog: added

Leonardo da Rosa authored 1 year ago and Matt Kasa committed 1 year ago

Pre-16.3 migration files were removed and schema
changes moved to db/init_structure.sql file

Changelog: changed

Changelog: added

Contributes to #440101

* Add sharding keys
* Define schemas

Changelog: other

This commit makes sure the reference FK for ci_stages
are now defined a the routing table level so all other new
partitions will inherit from it.

Changelog: other

Max Orefice authored 1 year ago and Marius Bobin committed 1 year ago

Changelog: other

Pam Artiaga authored 1 year ago and euko committed 1 year ago

To be used for this change: !146154

Sam Word authored 1 year ago and Tiger Watson committed 1 year ago

In PruneExpiredExportJobsService, EachBatch won't scale for large
volumes of data because we're trying to batch by pkey and a timestamp.
This introduces a new index on project_export_jobs to use loop-based
deletion and keyset pagination to more efficiently prune old uploads.