Initial multi-backend objectstorage client implementation


This MR introduces a gocloud.dev/blob-backed object storage client.

Based largely on this Claude generated plan: gitlab-org/quality/quality-engineering/team-tasks#4456 (comment 3466169269)

This MR:

  • Builds out a client model similar to our Postgres and Redis packages. (Config, Client, NewFromConfig)
  • Builds out a pluggable backend Provider pattern
  • Adds an initial S3 Provider implementation (OpenBucket)

A couple notable decisions:

  • Provider packages are built out into their own sub-packages. This is similar to the pattern gocloud.dev/blob uses; avoiding pulling in all backends as dependencies.
  • I added an in-memory S3 server (gofakes3 over httptest), mirroring the redistest helper, so package can test S3-backed code without a real bucket
  • Added an in-memory provider using "gocloud.dev/blob/memblob"
    • Was helpful while building out he provider model, but it's purpose might partially be covered by the above mock server.

Not in this MR:

  • Objectstorage configuration proto messages (To live in labkit-spec)
  • ResolveFromProto / methods for pulling config from attached secrets / known config locations.
  • Additional S3 authentication methods
  • Azure and GCS backends.
  • Logging, metrics, and tracing wired through each provider backend SDK
Edited by Luke Hollinda

Merge request reports

Loading
Loading