Skip to content

Normalize Maven upstream URLs by removing trailing slash

Tim Rizzirequested to merge
maven_trailing_url_bug into master

What does this MR do and why?

Problem

Creating upstreams with the same URL (one with a trailing slash, one without) creates duplicates targeting the same host, causing duplicate caching.

Solution

Added URL normalization in VirtualRegistries::Packages::Maven::Upstream that removes trailing slashes before validation. URLs like http://test.maven/ and http://test.maven are treated as the same.

Changes

  • Added before_validation :normalize_url callback to normalize URLs
  • Implemented normalize_url method using chomp('/') to remove trailing slashes
  • Only normalizes remote URLs (local URLs starting with gid:// are unchanged)
  • Added tests covering various URL formats and edge cases

Testing

Added test suite covering:

  • Removing trailing slashes from remote URLs
  • Handling URLs with paths
  • Preserving URLs without trailing slashes
  • Handling URLs with query parameters
  • Not modifying local URLs (gid://)
  • Preventing duplicate upstreams with different trailing slashes

References

Related to #579129

Screenshots or screen recordings

Before After

⚙️ How to set up and validate locally

Requirements:

  • Enable the feature flags: Feature.enable(:maven_virtual_registry) and Feature.enable(:ui_for_virtual_registries).
  • Have an EE license.
  • Have a top level group.
  1. Navigate to the top level group, then Deploy -> Virtual Registry.
  2. Create a maven registry.
  3. Create an Upstream with name upstream1 and url https://gitlab.com/maven/test1.
  4. Create an Upstream with name upstream2 and url https://gitlab.com/maven/test1/.

On master

Step (4.) is successful and we have now two upstreams with very similar urls = a chance to have duplications in cache entries = doubles the object storage usage.

With this MR

We get an error:

Group already has a remote upstream with the same url and credentials

which is the expected behavior 👍

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Fernandez

Merge request reports