Loading
Initial multi-backend objectstorage client implementation
This MR introduces a gocloud.dev/blob-backed object storage client.
Based largely on this Claude generated plan: gitlab-org/quality/quality-engineering/team-tasks#4456 (comment 3466169269)
This MR:
- Builds out a client model similar to our Postgres and Redis packages. (Config, Client, NewFromConfig)
- Builds out a pluggable backend Provider pattern
- Adds an initial S3 Provider implementation (OpenBucket)
A couple notable decisions:
- Provider packages are built out into their own sub-packages. This is similar to the pattern gocloud.dev/blob uses; avoiding pulling in all backends as dependencies.
- I added an in-memory S3 server (gofakes3 over httptest), mirroring the redistest helper, so package can test S3-backed code without a real bucket
- Added an in-memory provider using
"gocloud.dev/blob/memblob"- Was helpful while building out he provider model, but it's purpose might partially be covered by the above mock server.
Not in this MR:
- Objectstorage configuration proto messages (To live in labkit-spec)
ResolveFromProto/ methods for pulling config from attached secrets / known config locations.- Additional S3 authentication methods
- Azure and GCS backends.
- Logging, metrics, and tracing wired through each provider backend SDK
Edited by Luke Hollinda