Add Uninstrumented GCS ObjectStore provider

Luke Hollindarequested to merge
lh-objectstore-uninstrumented-gcs-provider into master

Parent Issue: gitlab-org/quality/quality-engineering/team-tasks#4456

This MR adds a GCS objectstore client, in a similar shape to the previously implemented S3 provider.

  • GCS Provider:
    • Constructs a native *storage.Client and hands it to gocloud's gcsblob.OpenBucket via Options.Client.
    • Self-registers via init().
  • Config:
    • Adds a JSON []bytes field (service-account key) to GCSCredentials.
    • Adds UniverseDomain
    • Extends Endpoint to cover GCS emulator/custom endpoints.
  • Credentials:
    • Use WithAuthCredentialsJSON(ServiceAccount, ...), additional authentication methods can be added as needed.
  • Testing:
    • Adds an in-memory GCS server backed by fsouza/fake-gcs-server

Not included in this MR:

  • Proto / environment config binding gcsConfigFromProto stays as a stub). This will be added in follow up MRs to Fairway / Labkit.

Note:

  • In some of the initial comments discussing AR's object storage requirements (AR is no longer a planned consumer of this package), they highlighted the need for reads to be through the JSON API. I myself don't understand the full differences between using the XML and JSON API; and have elected to set all reads to use the JSON API. (Based on AR's initial requirements and the fact that the SDK plans to make the JSON api the new default soon.)
    • This is an easy decision to revert. We can also quickly add a config option to toggle between the two if needed.

Merge request reports