Skip to content

Bump fog-aws to 3.3.0 and associated dependencies

Stan Hu requested to merge sh-bump-fog-gem into master

This fixes issues with AWS V4 signatures not working for Ceph S3, Digital Ocean Spaces, and OpenStack:

This merge request also introduces a monkey patch to squelch warnings from mixing Fog provider gems that use the old and new namespace format.

fog-core v2 changed the namespace format:

  • Old: Fog::<service>::<provider> (e.g. Fog::Storage::AWS).
  • New: Fog::<provider>::<service> (e.g. Fog::AWS::Storage)

To preserve backwards compatibility, fog-core v2.1.0 tries to load the old schema first, but falls back to the older version if that fails. This creates misleading warnings with fog-aws. See https://github.com/fog/fog-aws/issues/504#issuecomment-468067991 for more details.

fog-core v2.1.2 reverses the load order (https://github.com/fog/fog-core/pull/229), which works for fog-aws but causes a stream of deprecation warnings for fog-google. fog-google locked the dependency on fog-core v2.1.0 as a result (https://github.com/fog/fog-google/issues/421) until the new namespace is supported.

Since we currently have some Fog gems that have not been updated, this monkey patch makes a smarter decision about which namespace to try first. This squelches a significant number of warning messages, such as these warnings from fog-aws:

[fog][DEPRECATION] Fog::Storage::AWS is deprecated, please use Fog::AWS::Storage.
[fog][WARNING] Unrecognized arguments: region, aws_access_key_id, aws_secret_access_key, endpoint, aws_signature_version

Since this patch is mostly cosmetic, it can be removed safely at any time, but it's probably best to wait until the following issues are closed:

Test Plan

Test Provider Gem Pass?
Backup/restore Local fog-local ✅
Backup/restore Google fog-google ✅
Backup/restore AWS fog-aws ✅
Backup/restore Ceph S3 fog-aws ✅
Backup/restore DigitalOcean fog-aws ✅
Backup/restore Alibaba fog-aliyun ❓ Unable to sign up for Alibaba Object Storage. Apparently my credit card was deemed a security risk, and I'm not going to send my photo ID to China https://www.alibabacloud.com/help/doc-detail/69598.htm?spm=a273a.8138843.0.0.42b32458XTweEh.
Backup/restore OpenStack fog-openstack ✅
Object storage Local fog-local ✅ files are there but you can't actually serve out files this way :)
Object storage Google fog-google ✅
Object storage AWS fog-aws ✅
Object storage Ceph S3 fog-aws ✅
Object storage DigitalOcean fog-aws ✅
Object storage Alibaba fog-aliyun ❓ Unable to sign up for Alibaba Object Storage. Not sure if this even worked before because of https://gitlab.com/gitlab-org/gitlab-ce/issues/53144.
Object storage OpenStack fog-openstack ✅ for files that don't need URL encoding. https://gitlab.com/gitlab-org/gitlab-ce/issues/57714#note_145018285 points out a bug with some OpenStack providers.
Edited by Stan Hu

Merge request reports