Deleting remote branches via Geo secondary git proxy hangs when using SSH
This issue was separated out of #322519 (closed) as those are two completely independent issues.
This issue is valid for all the versions since Geo secondary git push proxy feature introduction.
If a user wants to delete a remote branch via a Geo secondary, they would normally type git push <REMOTE> --delete <BRANCH>
. When this is executed against a secondary node, via SSH - the command freezes up, and it is necessary for the user to press CTRL+C
to regain control. The branch however is deleted afterward.
There is no obvious errors in the server logs.
Steps to reproduce
Setup GitLab Geo with 1 primary node and 1 secondary node.
The example below assumes the following roles:
geo1 (geo1.vagrant) - primary
geo2 (geo2.vagrant) - secondary
-
Create a test project with 2 branches (
alpha
andbeta
). -
Clone the project locally with SSH. Setup remotes
geo1
andgeo2
. -
Run the following commands. Notice that the
alpha
branch is deleted when executed againstgeo1
, but when deleting thebeta
branch againstgeo2
, the command hangs. PressingCTRL+C
gets around the hang, and you can see that the branch has been deleted in GitLab.❯ git remote -v geo1 git@geo1.vagrant:root/delete-test.git (fetch) geo1 git@geo1.vagrant:root/delete-test.git (push) geo2 git@geo2.vagrant:root/delete-test.git (fetch) geo2 git@geo2.vagrant:root/delete-test.git (push) ❯ git fetch --all Fetching geo1 From geo1.vagrant:root/delete-test * [new branch] alpha -> geo1/alpha * [new branch] beta -> geo1/beta * [new branch] master -> geo1/master Fetching geo2 From geo2.vagrant:root/delete-test * [new branch] alpha -> geo2/alpha * [new branch] beta -> geo2/beta * [new branch] master -> geo2/master ❯ git push geo1 --delete alpha To geo1.vagrant:root/delete-test.git - [deleted] alpha ❯ git push geo2 --delete beta remote: remote: This request to a Geo secondary node will be forwarded to the remote: Geo primary node: remote: remote: git@geo1.vagrant:root/delete-test.git remote: ^C
What is the current bug behavior?
When deleting remote branches via a Geo secondary node, either the command hangs or doesn't complete successfully. As a result, the branch is deleted after pressing CTRL+C
(for SSH) or the command fails completely, and the branch is not deleted (HTTP).
What is the expected correct behavior?
When deleting remote branches via a Geo secondary node, the commands should be successful.
Geo SSH proxy push diagram
sequenceDiagram
participant Client
participant Secondary
participant SecondaryGitLabShell as Secondary gitlab-shell
participant Primary
Note over Client,Secondary: $ git push
Client->>SecondaryGitLabShell: ssh <single> git-receive-pack ''/repo.git''
SecondaryGitLabShell->>+Secondary: HTTP POST /api/v4/internal/allowed
Secondary-->>-SecondaryGitLabShell: OK, REDIRECT
SecondaryGitLabShell->>+Secondary: HTTP POST /api/v4/geo/proxy_git_push_ssh/info_refs
Secondary-->>Primary: HTTP GET /repo.git/info/refs?service=git-receive-pack
Primary-->>Secondary: <refs>
Secondary->>-SecondaryGitLabShell: <refs>
SecondaryGitLabShell->>+Secondary: HTTP POST /api/v4/geo/proxy_git_push_ssh/push
Secondary-->>Primary: HTTP POST /repo.git/info/refs?service=git-receive-pack/git-receive-pack
Primary-->>Secondary: <refs>
Secondary->>-SecondaryGitLabShell: <refs>
SecondaryGitLabShell-->>+Client: <refs>