Plan for moving issues from CE to EE as part of the single codebase migration

In https://gitlab.com/gitlab-org/gitlab-ee/issues/13304 we discuss a timeline and various steps to take to move to a single codebase. In this issue we will discuss one specific step: moving issues from CE to EE. We discuss this separately since there are several sub steps to take, potential hazards to consider, and the mentioned issue has trouble loading due to the large number of (system) notes.

Moving issues consists out of the following steps:

  1. Move the issue to gitlab-org/gitlab-ee.
  2. Create a note on the old issue explaining why it was moved.
  3. Lock the old issue to prevent new comments from being posted.

The order of these steps is deliberate. If we create the note in step two first, it would exist for both the old and new issue, which could be confusing. Locking the issue last is needed as otherwise @gitlab-bot might not be able to post any comments.

We have performed these steps for a limited number of issues created more than one year ago, using a script that uses the GitLab API. Using the API creates a lot of noise, as for every moved issue Emails are sent to all participants that have Email notifications enabled. We also create TODOs for the assignees of the moved issues. We also ran into issues where some issues could not be moved due to statement timeouts for queries that use the internal_ids table.

@marin and I talked about what steps we can take to keep the noise at a minimum. Not just for GitLab empoyees, but for all issue participants. To deal with this, we came up with the following procedure:

Instead of using the API, we use a Rails console to run a custom script. This script provides patched versions of the following classes:

  1. Issues::MoveService
  2. Issues::CreateService
  3. Issues::CloseService

Issues::MoveService will be patched so that it does not call its instance method notify_participants.

Issues::CreateService will be patched so that it does not create todos in its instance method after_create.

Issues::CloseService will be patched so that it does not create todos in its instance method close_issue.

We will then use this script to move a limited number of issues (<= 5 or so) older than one year, monitoring the effect this will have on the number of outgoing Emails and todos.

Once all is well and all of this has been announced we will start moving issues (older than one year) in batches of 1000. We will do this a few times until we are comfortable that we are not causing unnecessary pressure on GitLab.com, at which point we will start moving over all old issues.

Once all old issues have been moved over we will start moving all remaining issues, starting with the oldest ones first.

When all issues have been moved over, we will activate the gitlab-triage script created by @rymai in gitlab-org/quality/triage-ops!257 (merged). This script will ensure that newly created issues are moved over automatically.

Once all of this is done we can change the issue template for GitLab CE to clarify that new issues should be created in what is currently gitlab-org/gitlab-ee, which we will later rename to gitlab-org/gitlab.

TODO

  • Announce that we are going to move issues over
  • Write the Rails console script to move the old issues over, starting with a limited number of issues.
  • Move 5 000 old issues in batches of 1 000 each, monitoring GitLab between these batches
  • Move all remaining old CE issues
  • Move all remaining CE issues
  • Merge @rymai's gitlab-triage script Will do this after the rename
  • Update the gitlab-org/gitlab-ce issue template to explain where to create new issues
Edited by Yorick Peterse