Swap integer and bigint columns for notes and dependent tables for self-managed

This is the next step after Initialize the migration of notes.id and depend... (#389306 - closed). The process is outlined here.

These columns were already swapped for GitLab.com with #392815 (closed), now we have to do it for self-managed instances.

The columns we need to migrate are

• notes.id !127566 (merged)
• todos.note_id !125837 (merged)
• system_note_metadata.note_id !128803 (merged)
• epic_user_mentions.note_id !128797 (merged)
• suggestions.note_id !128892 (merged)
• issue_user_mentions.note_id !128934 (merged)
• note_diff_files.diff_note_id !128983 (merged)
• snippet_user_mentions.note_id !128991 (merged)
• design_user_mentions.note_id !129111 (merged)
• vulnerability_user_mentions.note_id !129082 (merged)
• commit_user_mentions.note_id !129117 (merged)
• merge_request_user_mentions.note_id !129134 (merged)
• timelogs.note_id !129258 (merged)
• award_emoji.awardable_id !129263 (merged)
• events.targe_id !129379 (merged)

changed milestone to %16.2

added Category:Database databaseactive devopsdata stores groupdatabase maintenancescalability missed:15.10 missed:15.11 missed:16.0 missed:16.1 priority1 sectioncore platform severity3 typemaintenance workflowin dev labels

assigned to @krasio

marked this issue as related to #392815 (closed)

added to epic &8070

removed missed:15.10 label

removed missed:15.11 label

removed missed:16.1 label

removed missed:16.0 label

@krasio In order to get a better understanding of our throughput and capacity, using issue weights based on merge requests. For this issue, would you consider how many merge requests it may take to implement this, add a comment enumerating them, and add the count as the issue weight? Thanks!

Examples (click to view)

An issue that just needs to update some documentation would be /weight 1 accompanied by a comment:

Just one merge request to documentation

An issue that might be broken out into separate merge requests for database, functional, documentation, and omnibus changes would be /weight 4 accompanied by a comment:

One to gitlab for database changes, one for new functionality, one for documentation changes, and one to omnibus

One MR per table, so 15.

set weight to 15

mentioned in merge request !125837 (merged)

changed the description

mentioned in issue gitlab-org/database-team/team-tasks#361 (closed)

mentioned in commit d660776a

mentioned in commit b879ee53

mentioned in commit e001b86a

changed milestone to %16.3

added missed:16.2 label

marked the checklist item todos.note_id !125837 (merged) as completed

@l.rosa @dfrazao-gitlab @praba.m7n I had a chat with @alexives about this one. It would be awesome if can merge 2 or 3 more MRs to for some of the other tables while I am on leave. Swapping the columns for self-managed is slightly more complicated that what we had to do for GitLab.com, as we have to consider the state of the instance (this depends on the release it was created from).

We can use !125837 (merged) as an example, I think all other cases will need the same checks. Diogo also has some experience with this as he had to do the same for merge_request_metrics.

If you manage to cover some of the other tables, this will be awesome, otherwise I'll continue when I get back from OOO. Thanks!

@Kras, sure! I believe we can work async through this issue. As we're start picking an item to work, we can report a note in this issue, like:

I'm starting working on:

• Item

+, we can pair and sync if the questions pop up along the way.

@l.rosa @dfrazao-gitlab @praba.m7n Thanks a lot! Feel free to send me the MRs for maintainer review.

assigned to @praba.m7n, @l.rosa, and @dfrazao-gitlab

I'm starting working on:

• notes.id

/cc @praba.m7n @dfrazao-gitlab @krasio

Working on epic_user_mentions.note_id

I am taking up

• snippet_user_mentions.note_id

changed the description

mentioned in merge request !128156 (merged)

mentioned in merge request !127566 (merged)

I'm starting working on:

• system_note_metadata.note_id

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed to address this issue

mentioned in merge request !128797 (merged)

mentioned in merge request !128803 (merged)

changed the description

changed the description

I'm starting working on:

• suggestions.note_id

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-2 to address this issue

mentioned in merge request !128892 (merged)

mentioned in commit 8c146ef9

I'm starting working on:

• issue_user_mentions.note_id

changed the description

changed the description

mentioned in commit 7098b8dd

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-3 to address this issue

mentioned in merge request !128983 (merged)

I am starting working on:

• note_diff_files

changed the description

changed the description

I am starting working on:

• snippet_user_mentions

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-4 to address this issue

mentioned in merge request !128991 (merged)

mentioned in merge request !128934 (merged)

changed the description

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-5 to address this issue

mentioned in merge request !129082 (merged)

I am starting working on:

• vulnerability_user_mentions

Merged

changed the description

Taking up

• design_user_mentions.note_id

created branch 417402-swap-integer-and-bigint-for-design_user_mentions-in-self-managed to address this issue

mentioned in merge request !129111 (merged)

changed the description

I am starting working on:

• commit_user_mentions.note_id

mentioned in issue gitlab-org/database-team/team-tasks#368 (closed)

changed the description

mentioned in merge request !129117 (merged)

mentioned in commit 0540ff60

I am starting working on:

• merge_request_user_mentions.note_id

mentioned in merge request !129134 (merged)

changed the description

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-8bd1a6b79592bd16 to address this issue

mentioned in merge request !129258 (merged)

I am starting working on:

• timelogs

changed the description

I am starting working on:

• award_emoji

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-7d86d094002f82b7 to address this issue

mentioned in merge request !129263 (merged)

changed the description

created branch 417402-swap-integer-and-bigint-columns-for-notes-and-dependent-tables-for-self-managed-4 to address this issue

mentioned in merge request !129379 (merged)

changed the description

marked the checklist item vulnerability_user_mentions.note_id !129082 (merged) as completed

marked the checklist item snippet_user_mentions.note_id !128991 (merged) as completed

mentioned in commit 0986396e

mentioned in commit 96625787

marked the checklist item epic_user_mentions.note_id !128797 (merged) as completed

mentioned in commit cfdf4526

marked the checklist item timelogs.note_id !129258 (merged) as completed

marked the checklist item commit_user_mentions.note_id !129117 (merged) as completed

mentioned in commit 86a83624

mentioned in merge request !129536 (closed)

mentioned in commit 8edb3efa

mentioned in commit 4b4892b4

marked the checklist item suggestions.note_id !128892 (merged) as completed

marked the checklist item system_note_metadata.note_id !128803 (merged) as completed

mentioned in commit a3dc04da

mentioned in issue gitlab-org/quality/engineering-productivity/master-broken-incidents#3341 (closed)

@dfrazao-gitlab @l.rosa @praba.m7n When writing test for the swap migrations, please make sure we clean up any column added during setup, otherwise we may end up with master:broken incidents like gitlab-org/quality/engineering-productivity/master-broken-incidents#3341 (comment 1518338296).

Thanks for fixing that Kras

marked the checklist item design_user_mentions.note_id !129111 (merged) as completed

mentioned in commit 2a06f11d

marked the checklist item note_diff_files.diff_note_id !128983 (merged) as completed

marked the checklist item events.targe_id !129379 (merged) as completed

mentioned in commit bf101a78

marked the checklist item issue_user_mentions.note_id !128934 (merged) as completed

changed milestone to %16.4

added missed:16.3 label

marked the checklist item award_emoji.awardable_id !129263 (merged) as completed

mentioned in commit b44244df

added twtriaged label

mentioned in commit 5de4e4ec

marked the checklist item merge_request_user_mentions.note_id !129134 (merged) as completed

changed the description

mentioned in commit ab535041

added workflowin review label and removed workflowin dev label

mentioned in commit 3e112281

added workflowstaging label and removed workflowin review label

marked the checklist item notes.id !127566 (merged) as completed

All MRs are merged, I think we can close this issue now. Thanks everyone!

We can start the clean up from 16.5.

closed

mentioned in issue gitlab-org/database-team/team-tasks#373 (closed)

mentioned in issue gitlab-org/database-team/team-tasks#379 (closed)

mentioned in issue #428250 (closed)

changed the description

added groupdatabase frameworks label and removed groupdatabase [DEPRECATED] label