Create virtual host configuration API for GitLab Pages
Description
In order to support the architecture updates for GitLab Pages, create an API so it is no longer necessary to load the virtual host configuration from a local file.
Proposal
This work has already started in this MR, finishing up that work will complete this issue.
Links / references
Original Pages API issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/58796
Related Epic: &1316 (closed)
Status update:
- Rename API::Internal class to API::Internal::Base gitlab-foss!32617 (merged)
- Extract Workhorse <-> GitLab JWT authentication to make it reusable gitlab-foss!32623 (merged)
- Add skeleton Pages internal API gitlab-foss!32732 (merged)
- Add support for custom domains to the Pages internal API gitlab-foss!32735 (merged)
- Add support for namespaces to Pages internal API !17197 (merged)
- Migrate Pages metadata on-demand !17204 (merged)
- Background migration to migrate Pages metadata !17993 (merged)
- omnibus-gitlab: Generate and symlink gitlab_pages_shared_secret omnibus-gitlab!3705 (merged)
Designs
- Show closed items
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Darby Frey added devopsrelease [DEPRECATED] + 1 deleted label
added devopsrelease [DEPRECATED] + 1 deleted label
- Darby Frey added to epic &1316 (closed)
added to epic &1316 (closed)
- Darby Frey mentioned in epic &1316 (closed)
mentioned in epic &1316 (closed)
- Darby Frey changed the description
changed the description
- Jason Yavorska added Category:Pages label
added Category:Pages label
- Vladimir Shushlin mentioned in issue gitlab-pages#161 (closed)
mentioned in issue gitlab-pages#161 (closed)
- Corrina Stasik changed milestone to %12.2
changed milestone to %12.2
- Author Developer
Marking gitlab-ce~992791 for %12.2
- Darby Frey added Deliverable label
added Deliverable label
- Darby Frey assigned to @krasio
assigned to @krasio
- Krasimir Angelov mentioned in commit omnibus-gitlab@7318df08
mentioned in commit omnibus-gitlab@7318df08
- Krasimir Angelov mentioned in merge request omnibus-gitlab!3444 (closed)
mentioned in merge request omnibus-gitlab!3444 (closed)
- Krasimir Angelov mentioned in commit omnibus-gitlab@3e173002
mentioned in commit omnibus-gitlab@3e173002
- Krasimir Angelov mentioned in commit omnibus-gitlab@951c2a1f
mentioned in commit omnibus-gitlab@951c2a1f
- Maintainer
Setting gitlab-ce
10690713 based on gitlab-ce4107696. - Maintainer
Setting gitlab-ce
10690713 based on gitlab-ce4107696. - 🤖 GitLab Bot 🤖 added 1 deleted label
added 1 deleted label
- Corrina Stasik added 1 deleted label
added 1 deleted label
- Contributor
@krasio @darbyfrey did this make %12.2?
Collapse replies - Author Developer
Still in progress, moving to %12.3
- Darby Frey changed milestone to %12.3
changed milestone to %12.3
- Maintainer
Status update - still have some work to do, MR is https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30787.
- Krasimir Angelov added workflowin dev label
added workflowin dev label
- Krasimir Angelov added workflowin review label and removed workflowin dev label
added workflowin review label and removed workflowin dev label
- 🤖 GitLab Bot 🤖 changed milestone to %12.3
changed milestone to %12.3
- Maintainer
Some update on MRs we need for this change:
- Rename API::Internal class to API::Internal::Base gitlab-foss!32617 (merged)
- Extract Workhorse <-> GitLab JWT authentication to make it reusable gitlab-foss!32623 (merged)
- Add skeleton Pages internal API gitlab-foss!32732 (merged)
- Add support for custom domains to the Pages internal API gitlab-foss!32735 (merged)
- Add support for namespaces to Pages internal API !17197 (merged)
- Migrate Pages metadata on-demand !17204 (merged)
- Background migration to migrate Pages metadata !17993 (merged)
- omnibus-gitlab: Generate and symlink gitlab_pages_shared_secret omnibus-gitlab!3705 (merged)
Edited by Krasimir Angelov 2 - Krasimir Angelov mentioned in commit gitlab-foss@5d1d5942
mentioned in commit gitlab-foss@5d1d5942
- Krasimir Angelov mentioned in commit gitlab-foss@e529360f
mentioned in commit gitlab-foss@e529360f
- Krasimir Angelov mentioned in commit gitlab-foss@eb914571
mentioned in commit gitlab-foss@eb914571
- Krasimir Angelov mentioned in commit gitlab-foss@ba73aef8
mentioned in commit gitlab-foss@ba73aef8
- Contributor
This item has now missed %12.3 and is being moved to %12.4 where it can be wrapped up. @darbyfrey this was another one waiting on maintainer review.
- Jason Yavorska changed milestone to %12.4
changed milestone to %12.4
- Jason Yavorska added missed-deliverable missed:12.3 labels
added missed-deliverable missed:12.3 labels
- Contributor
@jlenny would this be considered a direction item?
Collapse replies - Contributor
I don't think so @ogolowinski. Doesn't seem particularly high visibility.
Edited by Jason Yavorska
- Krasimir Angelov mentioned in commit gitlab-foss@54aa07d8
mentioned in commit gitlab-foss@54aa07d8
- Krasimir Angelov mentioned in commit gitlab-foss@d5de8b6e
mentioned in commit gitlab-foss@d5de8b6e
- Krasimir Angelov mentioned in commit gitlab-foss@66c8b7ac
mentioned in commit gitlab-foss@66c8b7ac
- Krasimir Angelov mentioned in commit gitlab-foss@b1dc49c9
mentioned in commit gitlab-foss@b1dc49c9
- Krasimir Angelov mentioned in commit b7110a59
mentioned in commit b7110a59
- Krasimir Angelov mentioned in merge request !17197 (merged)
mentioned in merge request !17197 (merged)
- Krasimir Angelov mentioned in commit f2ba88ed
mentioned in commit f2ba88ed
- Krasimir Angelov mentioned in commit 456e1527
mentioned in commit 456e1527
- Krasimir Angelov mentioned in merge request !17204 (merged)
mentioned in merge request !17204 (merged)
- Krasimir Angelov mentioned in commit c1bbc5c7
mentioned in commit c1bbc5c7
- Krasimir Angelov mentioned in commit 7a7868cc
mentioned in commit 7a7868cc
- Krasimir Angelov mentioned in commit b63e87a8
mentioned in commit b63e87a8
- Krasimir Angelov mentioned in commit 0e10dadd
mentioned in commit 0e10dadd
- Krasimir Angelov mentioned in commit 151a2b25
mentioned in commit 151a2b25
- Krasimir Angelov mentioned in commit c4008f6d
mentioned in commit c4008f6d
- Krasimir Angelov mentioned in commit ef7ca36f
mentioned in commit ef7ca36f
- Krasimir Angelov mentioned in commit 3a0ae19a
mentioned in commit 3a0ae19a
- Krasimir Angelov mentioned in issue #33106
mentioned in issue #33106
- Krasimir Angelov mentioned in commit 7a1294d6
mentioned in commit 7a1294d6
- Krasimir Angelov mentioned in commit 817c140e
mentioned in commit 817c140e
- Krasimir Angelov mentioned in commit 043df9b4
mentioned in commit 043df9b4
- Krasimir Angelov mentioned in merge request !17993 (merged)
mentioned in merge request !17993 (merged)
- Krasimir Angelov mentioned in commit 1e569fb0
mentioned in commit 1e569fb0
- Krasimir Angelov mentioned in commit 8442a947
mentioned in commit 8442a947
- Krasimir Angelov mentioned in commit gitlab-pages@f640b996
mentioned in commit gitlab-pages@f640b996
- Krasimir Angelov mentioned in commit b89a9909
mentioned in commit b89a9909
- Krasimir Angelov mentioned in commit 75b18413
mentioned in commit 75b18413
- Krasimir Angelov mentioned in commit 1bfe53a0
mentioned in commit 1bfe53a0
- Orit Golowinski added typefeature label
added typefeature label
- Krasimir Angelov mentioned in commit 4b7bffd9
mentioned in commit 4b7bffd9
- Krasimir Angelov mentioned in commit 748829de
mentioned in commit 748829de
- Maintainer
@ogolowinski In !17993 (merged) we introduced background migration. According to https://docs.gitlab.com/ee/development/background_migrations.html#when-to-use-background-migrations we need to announce it in the monthly release post along with an estimate of how long it will take to complete.
This is done in the Important notes on upgrading section - https://about.gitlab.com/handbook/marketing/blog/release-posts/#important-notes-on-upgrading.
Here is the text we need to include:
To help with the larger migrations we have introduced background migrations for this release. Background migrations are Sidekiq jobs that will run asynchronously. For GitLab.com these migrations took around 36 hours to complete, and no side effects were expected nor present.
To find the approximate time it will take to complete these migrations on your instance run the following command from a Rails console:
(Project.count.to_f / 300_000).ceil
. You can check the status of background migrations by running this command from a Rails console:Sidekiq::Queue.new('background_migration').size
.Let me know if you have any questions or if there's anything else I can help with.
/cc @darbyfrey
- Krasimir Angelov assigned to @ogolowinski
assigned to @ogolowinski
- Contributor
@jlenny Where should this go in the blog post? Also, should I mark it direction ?
Collapse replies - Contributor
@ogolowinski reading the description I can't tell what this actually does from a customer facing standpoint, so not sure. It sounds like maybe it's a backend thing so no, that wouldn't be included.
- Contributor
@jlenny please refer to Kras's comment above #28781 (comment 230957914) There is a migration involved
- Contributor
@ogolowinski it goes in the
important notes for upgrading
section as @krasio mentioned. Someone in the #release-post channel can help with how to edit that part. There's no need to mark this direction to include it there, though we should probably groom the description of this issue to be complete.
- Contributor
@krasio This will affect self managed customers , correct?
Collapse replies - Maintainer
@ogolowinski Yes but need to have significant number of projects in order to make this migration take more than few minutes.
- Darby Frey changed milestone to %12.5
changed milestone to %12.5
- Darby Frey added missed:12.4 label
added missed:12.4 label
- Contributor
@darbyfrey this has missed 12.4? If so, I need to remove the upgrade notes from the blog post, from what I understood GitLab.com has already migrated. @krasio can you confirm?
Collapse replies - Maintainer
@ogolowinski This is issue turned out bigger than we thought so I've been working on it in smaller chunks, you can see the progress here - #28781 (comment 217282591). So the issue itself as a whole is not going to make it for %12.4 but the migration from the blog post announcement will be there (and was already deployed to GitLab.com).
- Author Developer
@ogolowinski to add to @krasio's comment, there is one MR still is review that is preventing this issue from being fully closed. Aside from the migration, these changes won't have any customer impact in this release. Please keep the upgrade notes in the release post.
- Krasimir Angelov mentioned in commit omnibus-gitlab@1a6ae80f
mentioned in commit omnibus-gitlab@1a6ae80f
- Krasimir Angelov mentioned in commit omnibus-gitlab@1c762b60
mentioned in commit omnibus-gitlab@1c762b60
- Krasimir Angelov mentioned in merge request omnibus-gitlab!3705 (merged)
mentioned in merge request omnibus-gitlab!3705 (merged)
- Krasimir Angelov changed the description
changed the description
- Krasimir Angelov mentioned in commit omnibus-gitlab@f19c4cf6
mentioned in commit omnibus-gitlab@f19c4cf6
- Augustin Lemesle mentioned in commit Dalahro/gitlab@4d1a3d31
mentioned in commit Dalahro/gitlab@4d1a3d31
- Krasimir Angelov mentioned in commit gitlab-pages@313ed9f5
mentioned in commit gitlab-pages@313ed9f5
- Krasimir Angelov mentioned in merge request gitlab-pages!198 (merged)
mentioned in merge request gitlab-pages!198 (merged)
- Krasimir Angelov mentioned in commit gitlab-pages@1821232a
mentioned in commit gitlab-pages@1821232a
- Krasimir Angelov mentioned in commit gitlab-pages@e4daad77
mentioned in commit gitlab-pages@e4daad77
- Krasimir Angelov mentioned in commit omnibus-gitlab@f379fa83
mentioned in commit omnibus-gitlab@f379fa83
- Maintainer
This is almost there, we have the API built and released but last related MR (omnibus-gitlab!3705 (merged)) needs to wait for gitlab-pages!198 (merged) to be released for Pages so that we can safely merge.
Collapse replies - Contributor
@krasio gitlab-pages!198 (merged) has been merged, is omnibus-gitlab!3705 (merged) in the queue?
- Maintainer
Yes, it is but we have to wait for gitlab-pages!198 (merged) to be released first - omnibus-gitlab!3705 (comment 241177076).
- Contributor
Assigning @jmeshell who is the PM for this area.
- Orit Golowinski assigned to @jmeshell and unassigned @ogolowinski
assigned to @jmeshell and unassigned @ogolowinski
- Krasimir Angelov mentioned in merge request !19928 (merged)
mentioned in merge request !19928 (merged)
- 🤖 GitLab Bot 🤖 added missed:12.5 label
added missed:12.5 label
- 🤖 GitLab Bot 🤖 changed milestone to %12.6
changed milestone to %12.6
- Jackie Porter removed 1 deleted label
removed 1 deleted label
- Krasimir Angelov marked the checklist item omnibus-gitlab: Generate and symlink gitlab_pages_shared_secret omnibus-gitlab!3705 (merged) as completed
marked the checklist item omnibus-gitlab: Generate and symlink gitlab_pages_shared_secret omnibus-gitlab!3705 (merged) as completed
- Maintainer
Last piece was merged - omnibus-gitlab!3705 (merged).
The API is still behind a feature flag but I am going to close this issue now.
Edited by Krasimir Angelov - Krasimir Angelov closed
closed
- Krasimir Angelov added workflowproduction label and removed workflowin review label
added workflowproduction label and removed workflowin review label
- Krasimir Angelov mentioned in commit 2b8d2300
mentioned in commit 2b8d2300
- Krasimir Angelov mentioned in issue #39172 (closed)
mentioned in issue #39172 (closed)
- Krasimir Angelov mentioned in commit 7ac4d9f7
mentioned in commit 7ac4d9f7
- Krasimir Angelov mentioned in merge request !21529 (merged)
mentioned in merge request !21529 (merged)
- Krasimir Angelov mentioned in commit 8a9f5880
mentioned in commit 8a9f5880
- Krasimir Angelov mentioned in commit db0e44e6
mentioned in commit db0e44e6
- Krasimir Angelov mentioned in commit cd6ba78a
mentioned in commit cd6ba78a
- Krasimir Angelov mentioned in commit ba907f05
mentioned in commit ba907f05
- 🤖 GitLab Bot 🤖 mentioned in issue gl-retrospectives/release#16 (closed)
mentioned in issue gl-retrospectives/release#16 (closed)
- Marcel van Remmerden mentioned in commit 7816adee
mentioned in commit 7816adee
- Orit Golowinski added grouprelease [DEPRECATED] label and removed 1 deleted label
added grouprelease [DEPRECATED] label and removed 1 deleted label