GitLab can no longer import/pull mirror from "dumb" git servers

Summary

The addition of validation of git urls prevents the importing and pull mirroring of servers implemented via the dumb protocol.

The relevant merge request that brings in this breaking change is: !70165 (merged)

I'll copy in the comment I added to that MR:

I was not able to found any existing dumb protocol server in the wild and this will create even more code. Could be added later if needed

I've run into issues with this for the following servers (no doubt there are more, likely any cgit server with enable-http-clone enabled):

While it is good that you are validating that any server that advertises itself as a smart server follows the relevant requirements, ideally this would fall back to the dumb protocol.

To quote the section on 'Dumb clients' from the previously mentioned git http protocol documentation: "When examining the response clients SHOULD only examine the HTTP status code."

This is likely the cause of at least some of the issues reported here: #366769

Steps to reproduce

  • Create new repository
  • Select 'Import project'
  • Select 'Repository by URL'
  • Paste in 'https://git.tukaani.org/xz.git'
  • Observe the error message: "There is not a valid Git repository at this URL. If your HTTP repository is not publicly accessible, verify your credentials."

Example Project

This is a pull mirror created before the addition of the validation. Where mirroring is clearly working without issue: https://gitlab.com/freedesktop-sdk/mirrors/tukaani/xz

As a side effect of adding the validation, I can not change the user that mirrors the repository for this project as I get the same error message that it is not a valid URL, despite the URL already being set.

What is the current bug behavior?

Valid git repositories from servers implemented using the dumb protocol are rejected.

What is the expected correct behavior?

During validation, if a server does not report that it supports the smart protocol, GitLab should fall back to validating that the HTTP status code and allow the import/pull mirror to continue.

Relevant logs and/or screenshots

screenshot_2022-11-20-103334

Output of checks

This bug happens on GitLab.com

Possible fixes

https://gitlab.com/gitlab-org/gitlab/-/blob/7ca77d8ad5e1275748eb4d5e823c30468b568a78/app/services/import/validate_remote_git_endpoint_service.rb#L48

Edited by Ben Brown