Enabling pull mirroring with basic auth via REST API fails when username or password contains special characters
Summary
When pull mirroring is enabled for a project via the GitLab REST API and the remote repository is protected via basic auth, mirroring will fail if the provided username or password contain special characters such as @
or \
.
Steps to reproduce
- create a new project
- Try to add pull mirroring to it through the REST API by making a call similar to this one
curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
--url "https://gitlab.example.com/api/v4/projects/:id" \
--data "mirror=true&import_url=https://user@domain.org:PasswordWithSpecialCh@racters@gitlab.example.com/group/project.git"
where username and/or password contain a special character such as @
. For example user@domain.org
and PasswordWithSpecialCh@racters
- You will get a response like
{
"message": "Internal uri/invalid_uri_error error: bad URI(is not URI?): \"http://user@domain.org:PasswordWithSpecialCh@racters@gitlab.example.com/group/project.git\""
}
This is expected behavior, as GitLab uses a URI parser that implements RFC 3986.
- Apply URL encoding to your credentials and resubmit the request:
curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \
--url "https://gitlab.example.com/api/v4/projects/:id" \
--data "mirror=true&import_url=https://user%40domain.org:PasswordWithSpecialCh%40racters@gitlab.example.com/group/project.git"
- Authentication to the remote server will fail as the provided user name and password are not decoded before being sent to the remote server
What is the current bug behavior?
When pull mirroring is attempted with a remote repository protected by basic auth and the provided username or password contains special characters such as @ or , the mirroring fails with an "Internal uri/invalid_uri_error" message. When special characters are URL encoded, they are sent verbatim (and thus incorrectly) to the remote server for authentication.
What is the expected correct behavior?
The pull mirroring process should successfully authenticate and mirror the repository, even when the username or password contains special characters like @ or \
Results of GitLab environment info
Assessment of this bug was done locally in the current GDK-in-a-box setup, using an enterprise license. Executing the provided commands failed due to unknown user "git"
and gitlab-rake: command not found
.
Results of GitLab application Check
Assessment of this bug was done locally in the current GDK-in-a-box setup, using an enterprise license. Executing the provided commands failed due to unknown user "git"
and gitlab-rake: command not found
.
Possible fixes
Possible fixes would include URL decoding before making basic auth requests. However, this would mean changing behavior for existing repositories with pull mirroring whose basic auth credentials contain valid URL encodings. Another fix would be separating username and password into their own API parameters.