Container registry tags disappear after upgrading to 18.0.1 when using s3_v2 storage driver

Summary

After upgrading to GitLab 18.0.1, all container registry image tags disappear when using the s3_v2 storage driver. This causes a critical production issue as container images become inaccessible.

Steps to reproduce

  1. Configure GitLab container registry to use the s3_v2 storage driver with AWS S3
  2. Upgrade GitLab instance from version 17.11.x to 18.0.1
  3. Access the container registry through the UI or API
  4. Observe that all previously visible container tags are now missing

Example Project

This issue affects all projects using container registry with the s3_v2 storage driver after upgrading to 18.0.1.

What is the current bug behavior?

After upgrading to GitLab 18.0.1:

  • All container registry image tags disappear from the UI and API
  • Registry logs show errors related to s3_v2 storage driver with "operation error S3: HeadObject, serialization failed: serialization failed: input member Key must not be empty"
  • When rebooting the instance, tags appear briefly and then disappear again

What is the expected correct behavior?

  • Container registry image tags should remain visible and accessible after upgrading to GitLab 18.0.1
  • No errors should appear in the registry logs related to the S3 storage driver
  • The s3_v2 driver should work correctly as it did in previous versions

Relevant logs and/or screenshots

Registry error logs show the storage driver health check failing with serialization errors when attempting to access S3 storage.

Current registry configuration uses s3_v2 with standard parameters including bucket name, region, encryption, and chunksize settings.

Output of checks

This bug happens on self-managed GitLab instances.

Possible fixes

A temporary workaround is to change the storage driver from s3_v2 back to s3 in the registry configuration. This resolves the issue temporarily, but is not a long-term solution as the s3 driver is deprecated in GitLab 17.10 and will be removed in GitLab 19.0.

The issue appears to be a regression in the s3_v2 driver implementation in GitLab 18.0.1. Multiple users have reported the same issue in the feedback thread: #525855

Patch release information for backports

This is a critical issue affecting production environments and should be considered for an urgent patch release for GitLab 18.0.x.