Move the gitlab-docs project under the gitlab-org group

I don't remember why this project wasn't initiated under the gitlab-org group, but I think it's time we discussed about transferring it.

Pros:

  • Unified workflow: all boards under the gitlab-org roof, can create epics with issues related to both CE/EE and the gitlab-docs project

Cons:

  • ?

Considerations/Implications

  1. Registry - Are the Registry images migrated along with the project transfer?
  2. Pages - How about Pages? Will they continue to work?
    • Yes, BUT the DNS records will need to be changed if we're using CNAME.
  3. Labels - If we go with this we'd need to see where this is referenced and replace the URLs, and also find out if there will be any issues with the existing group labels that are under gitlab-com.
  4. Milestones - What happens with the milestones? Will the project inherit the milestones of gitlab-org?
  5. Epics - We have some epics at the gitlab-com group level https://gitlab.com/groups/gitlab-com/-/epics?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=Documentation. What would happen if we moved the project? It's fine if we need to manually log which issues were part of which epic and re-create the epics; we'd just need to know that, and if we’re supposed to delete the old epics manually before the move, to prevent breaking anything. (I know we do not support moving epics to another group.)
    • Epics are not transferred, but they still retain the issues even though they are now in a new group. We'll have to manually re-create them in the new group.

Impact

  1. During the transfer, we get no new deployments (hourly schedules)
  2. Review apps will not work until all Docker images are uploaded and the group name is changed in the script that deploys review apps in CE/EE/Runner projects.

Testing

  1. Create test project forked from gitlab-docs https://gitlab.com/gl-docsteam/gitlab-docs
    1. Make sure all version branches build correctly
    2. Make sure all pipelines build correctly (create: single Docker images, archive Docker image, latest Docker image)
  2. Create new group to migrate to https://gitlab.com/groups/gl-docsteam-new
  1. Create same milestone name in new and old group
  2. Create group labels in both groups.
    • One label with same name for both groups
    • One label only in old group
  3. Create an epic in the old group and attach an issue to it https://gitlab.com/groups/gl-docsteam/-/epics/1
  4. Remove all Registry images https://gitlab.com/gl-docsteam/gitlab-docs/container_registry
  5. Transfer project https://gitlab.com/groups/gl-docsteam-new
  6. Check that Pages still work (they worked after changing the DNS record)
  7. Change all paths for ALL branches: 
    find ./ -type f -exec sed -i 's/gl-docsteam\/gitlab-docs/gl-docsteam-new\/gitlab-docs/' {} \;
  8. Rebuild all registry images
  9. Check that Pages still work
  10. Check the builds still work (master, scheduled pipelines, single/archive/latest Docker images
  11. Check Registry migration https://gitlab.com/gl-docsteam/gitlab-docs/container_registry
  12. Check that Pages still work after rebuilding the site http://docs2.axilleas.me
  13. Test issue redirection https://gitlab.com/gl-docsteam/gitlab-docs/issues/1
  14. Check milestone migration
  15. Check label migration
  16. Check epic migration

You can see all my changes in https://gitlab.com/gl-docsteam/gitlab-docs/compare/cc8d66a146ba020e5268d77d38f8bdc809cfb884...master. I used a test site under my personal domain: http://docs2.axilleas.me (no https) using the Registry images uploaded in the forked project.

Before migration

  1. Migrate to group milestones (there's only %10.0) https://gitlab.com/gitlab-com/gitlab-docs/milestones?sort=due_date_desc&state=all

  2. Make .gitlab-ci.yml project-agnostic by providing CI_REGISTRY_IMAGE instead of hardcoded paths. We can do this right away as it doesn't affect the existing project.

  3. Have an MR ready to be merged (but not merge!) as soon as the transfer is done for the following projects:

    Do this by replacing the occurrences of gitlab-com/gitlab-docs to gitlab-org/gitlab-docs:

    find ./ -type f -exec sed -i 's/gitlab-com\/gitlab-docs/gitlab-org\/gitlab-docs/' {} \;

  4. Get a production engineer on-call for the next 30min to change the DNS record and solve anything else

  5. Remove all Registry images

  6. Disable the scheduled pipeline

  7. Transfer project

Immediately after migration

  1. Change the DNS CNAME to point to the new group https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7235 Not needed, we're using an A record
  2. Merge the gitlab-docs MR of the name change and cancel the pipeline as it will fail since there are no Registry images at this point !489 (merged)
  3. Rebuild all the helper Registry images:
    1. Run a manual job of image:bootstrap. This will build registry.gitlab.com/gitlab-org/gitlab-docs:bootstrap. Wait to upload the image.
    2. Run a manual job of image:builder-onbuild. This will build registry.gitlab.com/gitlab-org/gitlab-docs:builder-onbuild.
    3. Run a manual job of image:nginx-onbuild. This will build registry.gitlab.com/gitlab-org/gitlab-docs:nginx-onbuild.
  4. Rebuild all the versions Registry images:

    1. The versions will be built by running a pipeline of the related branches/versions. This will be done by pushing to all respective branches:

      #!/bin/sh

array=( 10.3 10.4 10.5 10.6 10.7 10.8 11.0 11.1 11.2 11.3 11.4 11.5 11.6 11.7 11.8 11.9 11.10 11.11 12.0 )

for REF in "${array[@]}"
do
  git checkout $REF
  # Make sure we're on the right branch
  [ "$(git rev-parse --abbrev-ref HEAD)" == "$REF" ] || exit 1
  find ./ -type f -exec sed -i 's/gitlab-com\/gitlab-docs/gitlab-org\/gitlab-docs/' {} \;
  git status
  echo
  echo "=> The above files have changed. Are you sure you want to commit and push? [N/y]"
  read answer
  [ "$answer" == "y" ] || (git checkout . ; exit 1)
  git add .
  git commit -m "Change project group to 'gitlab-org'"
  git push origin $REF
done

    2. Enable the scheduled pipeline and run it. This will build:

      registry.gitlab.com/gitlab-org/gitlab-docs:archives
registry.gitlab.com/gitlab-org/gitlab-docs:latest
  5. Merge the MRs of the name change for the following projects:
  6. Although GitLab automatically supports redirects, let's find and replace all occurrences of gitlab-com/gitlab-docs. Create MRs for other projects:
  7. Update our boards?

Checks

  1. Check that Pages still work
  2. Check the builds still work (master, scheduled pipelines, single/archive/latest Docker images
  3. Check Registry migration https://gitlab.com/gitlab-org/gitlab-docs/container_registry
  4. Check that Pages still work after rebuilding the site https://docs.gitlab.com
  5. Test issue redirection https://gitlab.com/gitlab-com/gitlab-docs/issues/310
  6. Check milestone migration
  7. Check label migration
  8. Check epic migration

Lastly:

  1. Announce in team call
  2. How much of this process is documented? Make sure we update our docs with our findings.
Edited by Achilleas Pipinellis