### Problem to solve <!-- What problem do we solve? --> Some customers automation scripts are broken because in some cases "git push" using deploy keys is not working any more. This happened, for examples, for deploy keys that were created by users, whose accounts are blocked now or keys having null user_id in the database. ### Intended users <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ --> ### Further details <!-- Include use cases, benefits, and/or goals (contributes to our vision?) --> #### Use case I created 4 ssh keys, added them as global from admin interface. In the database table "keys" they had "user_id" set to my user's id. I left first as is, for others set "user_id" field to: NULL, id of blocked user, id of another user accordingly. When I ssh to the gitlab instance with this keys I get greetings like this: * key1: Welcome to GitLab, @[my_user] * key2: Welcome to GitLab, Anonymous! * key3: Welcome to GitLab, @[blocked_user] * key4: Welcome to GitLab, @[other_user] I find it strange also, that deploy key is linked in such a way with its creator, but this is offtopic here. Then I prepared a private project: created an empty project, enabled those depoy keys in it, set write permission, cloned, made initial commit, pushed. After that tested pull/push operations with different deploy keys. Clone or pull works with all those keys. Push operation works only with the first key, for other keys I receive such errors: * key2 (null user) ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: This action cannot be performed by internal users To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` * key3 (blocked user) ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: Your account has been blocked. To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` * key4 (other user): ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: The project you were looking for could not be found. To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` When I was dealing with one of colleagues issues, I was able to push with key "beloning" to other user, who had Developer permission for the repository. Do not know what type of access other user needs in this case, have not tested all the cases. #### Proposed workaround that works for some users: > When I noticed that file don't add keys, I delete file /var/opt/gitlab/.ssh/authorized_keys and stared command sudo gitlab-ctl reconfigure. > > This file create. Via web-interface I delete all keys, and add their again. > > It worked! ### Proposal <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> As per https://gitlab.com/gitlab-org/gitlab/issues/35779#note_250710439 there is a workaround offered which basically queries for expired users and replaces the user with a valid user while maintaining the deploy key - so that in fact no project configuration needs to change since the deploy key stays the same. Under the admin area->deploy keys currently allows you to create instance-wide SSH deploy keys. Can we add a list there of the anomalies - meaning display a table only for blocked and expired users | Project name | Project ID | User status | User ID | Deploy Key | |----|--|--|--|---| The admin can edit the user ID from this console and this will make the change in the DB without needed to run anything.  Also we would need to capture this in an audit log ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?--> There are scenarios, where the blocked user still has/owns the private SSH key which pairs with the public SSH key imported into the project. Would not that raise a security concern? ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html Add all known Documentation Requirements here, per https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements If this feature requires changing permissions, this document https://docs.gitlab.com/ee/user/permissions.html must be updated accordingly. --> ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> ### What is the type of buyer? <!-- Which leads to: in which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ### Links / references * https://stackoverflow.com/questions/29749485/dont-work-ssh-in-gitlab/29765704#29765704 /label ~feature ### Old issue for safe keeping <details> ### Summary After updating from version 12.3.x to 12.4.1, I'm getting complaints from colleagues that they have their automation scripts broken because in some cases "git push" using deploy keys is not working any more. This happened, for examples, for deploy keys that were created by users, whose accounts are blocked now or keys having null user_id in the database. I have also find similar issue here: https://forum.gitlab.com/t/unable-to-push-to-repo-after-updating-gitlab/31088 ### Steps to reproduce I created 4 ssh keys, added them as global from admin interface. In the database table "keys" they had "user_id" set to my user's id. I left first as is, for others set "user_id" field to: NULL, id of blocked user, id of another user accordingly. When I ssh to the gitlab instance with this keys I get greetings like this: * key1: Welcome to GitLab, @[my_user] * key2: Welcome to GitLab, Anonymous! * key3: Welcome to GitLab, @[blocked_user] * key4: Welcome to GitLab, @[other_user] I find it strange also, that deploy key is linked in such a way with its creator, but this is offtopic here. Then I prepared a private project: created an empty project, enabled those depoy keys in it, set write permission, cloned, made inital commit, pushed. After that tested pull/push operations with different deploy keys. Clone or pull works with all those keys. Push operation works only with the first key, for other keys I receive such errors: * key2 (null user) ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: This action cannot be performed by internal users To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` * key3 (blocked user) ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: Your account has been blocked. To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` * key4 (other user): ``` Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Writing objects: 100% (3/3), 256 bytes | 256.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0) remote: GitLab: The project you were looking for could not be found. To ssh://gitlab/user/test-keys.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to 'ssh://git@gitlab/user/test-keys.git' ``` When I was dealing with one of colleagues issues, I was able to push with key "beloning" to other user, who had Developer permission for the repository. Do not know what type of access other user needs in this case, have not tested all the cases. ### What is the current *bug* behavior? Deploy keys that were working on the 12.3.x version, are no longer working and I see no reason why user, that created them should affect its behaviour. Because now "anonymous" keys are not working, as keys, created by some categories of users. And I see no work case how to make a deploy key, that will work independently of the status of some account. ### What is the expected *correct* behavior? It should be possible to use deploy key with write access, that was added by blocked or some unrelated user. A working script should continue working if the deploy key is still enabled in the repo, no matter that some user was blocked. Because currently somebody have to add the key for the script to work, but later it can stop working on the project and that should not break it. ## Proposal Create a way that will replace an existing deploy key with a new one in the UI with the correct permissions (will in fact replace the entry in the DB), that will in turn change the keys for the existing projects </details>