Investigate: Allow "moving" a project that has container registry data between groups; under the same namespace
🌱
Context This issue builds on &9459 to identify a (high-level) approach to iteratively solution project move operations (within a namespace) where projects with container repositories are involved.
Problem
🚒
Problem The inability to move a project with registry data to a new group/namespace has been a known limitation of the GitLab Container Registry since its MVC release in %8.8. Right now, attempting to move a project that has non-empty (at least one tag) container repositories leads to an error.
The current workaround is to delete all tags and only then can a project can be moved to the desired group/namespace. This is not user-friendly and is inefficient as it requires users to do the manual labor of :
- Temporarily storing their container repository contents while they go about deleting the images in their "old" repo
- Re-uploading the temporary stored repository content to the "newly moved" location.
In this issue we will focus primarily on the problem of moving a project between groups under the same *"namespace", as there will be more complexity (e.g security) associated with moving projects across groups in top level namespaces
For the purpose of this epic we will use the following definitions:
- A "namespace" is the roo of a project/repository without the (e.g the namespace for
my-top-level-namespace/my-group/my-project
ismy-top-level-namespace
) - A "group" is the immediate path before a project's name in the path a project's path (e.g the group for
my-top-level-namespace/my-group/my-project
ismy-top-level-namespace/my-group
)
Reproduction
🔭
Steps to reproduce - Create a project
my-project
in groupmy-group1
under top-level-namespacemy-top-level-namespace
. We'll assume the project's full path ismy-top-level-namespace/my-group1/my-project
; - Tag and push an image to
registry.domain.com/my-top-level-namespace/my-group1/my-project
; - Tag and push another image to two sub-repos (i.e
sub-repository1
andsub-repository2
) under the same base repo bath ofmy-top-level-namespace/my-group1/my-project
; - Attempt to move the project
my-project
frommy-group1
group to a new groupmy-group2
. This fails with the messageProject cannot be transferred, because tags are present in its container registry
Rails
⏳
Current Rails Flow (that surfaces to the problem) TBD
Ideal Scenario
🍏
Ideal Scenario When a user requests a "project move" within groups in the same top-level-namespace; the container registry repositories for the corresponding project should be moved to the desired group unless:
- A. There will be conflicting container repositories "paths" that result from the move
- B. The user does not have the necessary access roles for both the "source" namespace and the "target" namespace to perform a move
- C There will be storage limitation constraints (e.g insufficient group storage) that arise from the move (
🤔 is this possible?)
Other than the cases above a project move should succeed.
In cases where the "target" group already contains container repositories (or sub-repositories) (that do not result in A, B or C above); we should proceed to add the source project's repositories to the existing target group's existing repositories.
Building from the example namespace provided in the Steps to reproduce section, we'll assume a scenario where a user wants to move a project my-project
to group my-group1
in namespace my-top-level-namespace
with container repositories my-top-level-namespace/my-group1/my-project
, my-top-level-namespace/my-group1/my-project/sub-repository1
and my-top-level-namespace/my-group1/my-project/sub-repository2
i,e:
graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-group1[my-group1] -.->| move | my-project[my-project]
my-project -->| | sub-repository1[sub-repository1]
my-project -->| | sub-repository2[sub-repository2]
to:
-
A: an existing group
my-group2
with no sub-repositories (in the same namespace top-level-namespace) -
B: an existing group
my-group3
with non-conflicting sub-repositories (in the same namespace top-level-namespace) -
C: an existing group
my-group4
with conflicting sub-repositories (in the same namespace top-level-namespace) -
D: a non existing namespace
my-group5
(in the same namespace top-level-namespace)
my-project
from my-group1
to my-group2
with no sub-repositories
A: Move graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-top-level-namespace[my-top-level-namespace] -->| | my-group2[my-group2]
my-group1[my-group1] -.-| detach | my-project[my-project]
my-group2[my-group2] -.->| attach | my-project[my-project]
my-project -->| | sub-repository1[sub-repository1]
my-project -->| | sub-repository2[sub-repository2]
my-project
from my-group1
to my-group3
with non-conflicting sub-repositories
B: Move graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-top-level-namespace[my-top-level-namespace] -->| | my-group3[my-group3]
my-group1[my-group1] -.-| detach | my-project[my-project]
my-group3[my-group3] -->| | my-project2[my-project2]
my-group3[my-group3] -.->| attach | my-project[my-project]
my-project2 -->| | sub-repository1[sub-repository1]
my-project2 -->| | sub-repository2[sub-repository2]
my-project -->| | sub-repository1*[sub-repository1]
my-project -->| | sub-repository2*[sub-repository2]
my-project
from my-group1
to my-group4
with conflicting sub-repositories
C: Move graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-top-level-namespace[my-top-level-namespace] -->| | my-group4[my-group4]
my-group1[my-group1] -->| fail detach | my-project[my-project]
my-project -->| | sub-repository1[sub-repository1]
my-project -->| | sub-repository2[sub-repository2]
my-group4[my-group4] -.->| fail attach | my-project[my-project]
my-group4[my-group4] -->| | my-project*[my-project]
my-project* -->| | sub-repository1*[sub-repository1]
my-project* -->| | sub-repository2*[sub-repository2]
my-project
from my-group1
to a non existing namespace/group my-group5
D: Move graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-top-level-namespace[my-top-level-namespace] -->| create new | my-group5[my-group5]
my-group1[my-group1] -.-| detach | my-project[my-project]
my-group5[my-group5] -.->| attach | my-project[my-project]
my-project -->| | sub-repository1[sub-repository1]
my-project -->| | sub-repository2[sub-repository2]
Registry
📦
Registry "Repository Move" Operation There is currently no support for moving a container repository on the registry side once a project move has been initiated.
On the registry database, a repository is represented as a row with a : path
, name
, id
and top_level_namespace_id
attribute. where:
-
path
is the globally unique full path of the container repository (e.gmy-top-level-namespace/my-group1/my-project/sub-repository1
) -
name
is the last path segment ofpath
(e.gsub-repository1
formy-top-level-namespace/my-group1/my-project/sub-repository1
). -
id
is the globally unique container repository identifier`. -
top_level_namespace_id
is a globally unique identifier for the the root namespace in a container repository path (e.g the top level namespace formy-top-level-namespace/my-group1/my-project/sub-repository1
ismy-top-level-namespace
and thus has a unique id)
So, to facilitate a valid project "move" (as shown in the Ideal Scenario section) we would need to update:
- The
path
for the root repository (if any) and all sub-repositories of the source project (I.e the project to be moved), so that:- The
path
begins with the target group's absolute path suffixed by the root repository name
- The
In the simple example of moving a project my-project
from my-group1
in top level namespace of my-top-level-namespace
to my-group-2
in the same namespace
This translates to detaching the my-group1
project branch and re-attaching the branch to my-group-2
:
graph TD
my-top-level-namespace[my-top-level-namespace] -->| | my-group1[my-group1]
my-top-level-namespace[my-top-level-namespace] -->| | my-group2[my-group2]
subgraph
my-group1[my-group1] -.-| detach | my-project*[my-project]
my-group2[my-group2] -.->| attach | my-project*[my-project]
my-project* -->| | sub-repository1*[sub-repository1]
my-project* -->| | sub-repository2*[sub-repository2]
end
The proposed solution should account for
-
Auth Validation -
New gitlab/v1/
endpoint-
Race conditions (in a case where two move/rename requests targeting the same projects are received) -
Dry run to validate the potential of moving a repository
-