Extract common parts of the Maven dependency proxy

Context

In Maven dependency proxy (&3610 - closed), we implemented and delivered the Maven dependency proxy. Read about it here: https://docs.gitlab.com/ee/user/packages/package_registry/dependency_proxy/.

We are now starting working on npm Virtual Registry MVC: Single Upstream and A... (&3608) which is exactly the same features set (GitLab acting as a pull through cache) but for the NPM package format.

The Maven dependency proxy has some interactions with the Package Registry as it uses it as a cache location (see https://docs.gitlab.com/ee/user/packages/package_registry/dependency_proxy/#advanced-caching). This is the core behavior of the dependency proxy for packages that we will need to implement for NPM.

To avoid a massive code duplication, in this MR, we're going to extract all the common logic that will be used by both dependency proxies.

This is preparatory work for the implementation of NPM dependency proxy: implement the download tg... (#435644 - closed) and later.

If you're interested to see these changes + the first step for the NPM proxy, see NPM dependency proxy tgz download file endpoint (!144320 - merged). If we look at the ee/lib/api/dependency_proxy/packages/npm.rb, we only need 100 lines to implement the first endpoint of the NPM dependency proxy

What does this MR do and why?

• Extract the Maven dependency proxy caching logic into helper functions.
• Update the tracking events so that they adapt to the considered package format.
• Move the DependencyProxy::Packages::Maven::VerifyPackageFileEtagService to DependencyProxy::Packages::VerifyPackageFileEtagService as this service is part of the caching logic and is not really tied to a package format (and we're going to use for NPM too).
  • Move the related specs.

No changelog here as this is a pure code extract with no impact on the existing Maven dependency proxy behaviors. If there is an impact, that will be considered as a typebug.

We are quite confident in this refactoring as the maven dependency proxy is covered by a pretty large requests spec and a feature spec (which also checks all the s done with workhorse).

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

No changes in the UI

How to set up and validate locally

Follow https://docs.gitlab.com/ee/user/packages/package_registry/dependency_proxy/

ee/app/services/dependency_proxy/packages/maven/verify_package_file_etag_service.rb

-# frozen_string_literal: true
-
-module DependencyProxy
-  module Packages
-    module Maven
-      class VerifyPackageFileEtagService
-        include ::Gitlab::Utils::StrongMemoize
-
-        REGISTRY_NOT_AVAILABLE_ERROR_CODE = 503
-        REGISTRY_NOT_AVAILABLE_MESSAGE = 'External registry is not available'
-        TIMEOUT = 5
-
-        def initialize(remote_url:, package_file:)
-          @remote_url = remote_url
-          @package_file = package_file
-        end
-
-        def execute
-          return ServiceResponse.error(message: 'invalid arguments', reason: :invalid_arguments) unless valid?
-          return error_with_response_code unless response.success?
-          return ServiceResponse.error(message: 'no etag from external registry', reason: :no_etag) if etag.blank?
-          return ServiceResponse.success if etag_match?
-
-          ServiceResponse.error(
-            message: "etag from external registry doesn't match any known digests",
-            reason: :wrong_etag
-          )
-        rescue *::Gitlab::HTTP::HTTP_ERRORS
-          error_with_response_code(
-            code: REGISTRY_NOT_AVAILABLE_ERROR_CODE,
-            message: REGISTRY_NOT_AVAILABLE_MESSAGE
-          )
-        end
-
-        private
-
-        attr_reader :remote_url, :package_file
-
-        def response
-          ::Gitlab::HTTP.head(remote_url, follow_redirects: true, timeout: TIMEOUT)
-        end
-        strong_memoize_attr :response
-
-        def etag_match?
-          return false unless sanitized_etag
-
-          # Remote registries can have different ways to return the ETag field:
-          # GitLab: md5
-          # Maven Central: md5
-          # Artifactory: sha1
-          # Github: No ETag field
-          # Sonatype Nexus: custom string with the sha1. Example {SHA1{e9702caacd0b915b0495f6d191371d61c379cd1c}}.
-          #
-          # We thus need to check the etag field in different ways.
-
-          # Check if the Etag is exactly one of the digests.
-          return true if %i[md5 sha1 sha256].any? { |digest| sanitized_etag == package_file["file_#{digest}"] }
-          return true if %i[sha1].any? { |digest| sanitized_etag.include?(package_file["file_#{digest}"]) }
-
-          false
-        end
-
-        def sanitized_etag
-          return unless etag.present?
-
-          etag.delete('"')
-        end
-        strong_memoize_attr :sanitized_etag
-
-        def etag
-          strong_memoize_with(:etag, response) do
-            response.headers['etag']
-          end
-        end
-
-        def valid?
-          remote_url.present? && package_file
-        end
-
-        def error_with_response_code(code: response.code, message: nil)
-          message ||= "Received #{code} from external registry"
-          Gitlab::AppLogger.error(
-            service_class: self.class.to_s,
-            project_id: package_file.package&.project_id,
-            message: message
-          )
-
-          ServiceResponse.error(message: message, reason: :response_error_code)
-        end
-      end
-    end
-  end
-end