Loading
Commits on Source 59
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
Provider implementations will live is subpackages, to avoid pulling in heavy cloud sdk dependencies when the user may only want to use one backend
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
-
Luke Hollinda authored
Co-authored-by:GitLab Duo <gitlab-duo@gitlab.com>
-
Elliot Forbes authored
Initial multi-backend objectstorage client implementation See merge request !546 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Luke Hollinda <lhollinda@gitlab.com>
-
Florian Forster authored
Adds an `Addrs` method to the Redis `authnFormat` interface so client addresses can be delivered through the mounted secret rather than only the infrastructure proto. `configFromProto` now calls `pf.Addrs` instead of reading `cfg.GetAddrs()` directly. When the proto carries addresses they are used as-is; otherwise `authenticatedFormat` reads the secret key named by the projection's `addrs` field and parses a comma-separated `<host>:<port>` list, trimming whitespace around each entry. `noSecretFormat` continues to return only the proto addresses. This supports provisioned endpoints whose address is not known at config time and arrives via the secret alongside the credentials. Extracts a `config()` helper for the shared nil-infra / nil-Redis checks used by both `AuthConfig` and `Addrs`, plus a `resolveSecret` helper that resolves an arbitrary projection field through the secret provider. Bumps `fairway` to v1.30.0 for the `addrs` projection field. Adds table tests for both formats covering proto passthrough, single and comma-separated secret values, whitespace trimming, and the missing provider, missing projection field, and key-not-found error paths. Issue: gitlab-com/gl-infra/platform/runway/team#931
-
Florian Forster authored
Previously, splitting the secret value on commas kept empty entries, so a value with stray, doubled, or trailing commas yielded empty address strings that were passed to the Redis client as invalid addresses. Empty entries are now dropped after trimming, and a value that resolves to no addresses returns the new `ErrEmptyAddresses` sentinel with the raw value included for debugging. Adds tests for stripping interleaved empty entries and for the all-empty value returning an error. Issue: gitlab-com/gl-infra/platform/runway/team#931
-
Florian Forster authored
Adds `ProjectionProvider`, a new `Provider` implementation that resolves secret paths by reading field values from a protobuf message rather than hardcoding key names. `NewProjectionProvider` wraps an underlying `Provider` with a `proto.Message` whose string fields map logical credential names (e.g. `"username"`, `"password"`) to the actual key names stored in the Secret. `Get` looks up the requested name against the message descriptor by text name, rejects non-string fields with `ErrProjectionInvalid`, and returns `ErrProjectionMissing` when an optional field is absent — distinguishing internal logic errors from user configuration errors so callers can surface the right message. Adds `TestProjection` covering successful lookups against `PostgreSQLProjection` and the missing-field path, and `TestProjection_internalErrors` covering a non-string field (`port`) and a field name that does not exist in the descriptor.
-
Florian Forster authored
-
Florian Forster authored
-
Florian Forster authored
Merges `TestProjectionProvider` and `TestProjectionProvider_internalErrors` into a single table-driven test, covering successful lookups, unset and empty fields (`ErrProjectionMissing`), a non-existent field name, and a non-string field type (both `ErrProjectionInvalid`). Adds `t.Parallel()` at both the top-level and sub-test level.
-
Florian Forster authored
Adds `PrefixProvider`, a thin `Provider` decorator that prepends a fixed string to the secret name before delegating to an underlying `Provider`. The prefix is applied verbatim, so callers that want path semantics must include a trailing slash (e.g. `"postgresql/"`).
-
Florian Forster authored
Replaces the inline secret-resolution logic in `projectionFormat` with the generic `secret.ProjectionProvider` and `secret.PrefixProvider` types introduced earlier. `newProjectionFormat` now resolves the `PostgreSQLProjection` upfront (returning an error immediately on invalid config) and stores a pre-built `secret.Provider` chain — `PrefixProvider("postgresql/", ...)` wrapping a `ProjectionProvider` — on the struct. `resolveSecret` and `resolveConfigKey` are simplified to delegate directly to that provider, removing the inline path-assembly, the `strings.Contains` path-traversal guard (now redundant since `ProjectionProvider` resolves the key name from the proto descriptor), and the `secretRef *string` parameter from `resolveConfigKey`. Changes `newPSQLFormat` to return `(psqlFormat, error)` so that `projection()` failures (unknown format, missing CUSTOM projection) are surfaced before `DSN` is called. `client.go` and `client_test.go` are updated accordingly. The `hostArgs` and `portArgs` struct fields are renamed from `*Conf`/`*Proj` to `*Config`/`*Secret` for clarity. Replaces the package-local `ErrSecretProjectionMissing` sentinel with `secret.ErrProjectionMissing` from the shared `secret` package. -
Florian Forster authored
-
Florian Forster authored
-
Florian Forster authored
Renames the `secretProvider` field and constructor parameter to `base`, and the `Get` parameter from `secretName` to `fieldName`. The old name was misleading: the argument is a field in the projection proto, not a secret name. The local holding the resolved value is renamed to `secretName`, which is what it actually is. Documents that `fieldName` is matched against the projection proto's field JSON names (camelCase) since users configure these via YAML, and links the `ErrProjectionMissing` and `ErrProjectionInvalid` sentinels in the doc comment.
-
Luke Hollinda authored
feat(secret): Add proto-driven `ProjectionProvider` provider for secret key mapping. See merge request !553 Merged-by:
Luke Hollinda <lhollinda@gitlab.com> Approved-by:
Luke Hollinda <lhollinda@gitlab.com> Reviewed-by:
Florian Forster <fforster@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Luke Hollinda authored
feat(secret): Add `PrefixProvider` to prepend a path prefix to secret names. See merge request !554 Merged-by:
Luke Hollinda <lhollinda@gitlab.com> Approved-by:
Luke Hollinda <lhollinda@gitlab.com> Reviewed-by:
GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Luke Hollinda authored
feat(redis): Resolve client addresses from the secret projection. See merge request !551 Merged-by:
Luke Hollinda <lhollinda@gitlab.com> Approved-by:
Matthew J. Hoad <mhoad@gitlab.com> Approved-by:
Luke Hollinda <lhollinda@gitlab.com> Reviewed-by:
GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Florian Forster authored
-
Florian Forster authored
Adds `New(ctx, ...Option)` as the standard entry point for creating an `objectstore.Client` from the runtime infrastructure config, mirroring the pattern used in the `postgres` package. It loads `infrastructure.DefaultConfig()` when no `WithInfraConfig` option is provided, then delegates to `configFromProto` before calling the existing `NewWithConfig`. `configFromProto` reads the `ObjectStore` oneof from the proto config and dispatches to `gcsConfigFromProto` or `s3ConfigFromProto`. It wraps the infrastructure secret provider in a `PrefixProvider("objectstore/")` so that all downstream secret lookups are automatically namespaced. Metadata fields (`name`, `namespace`, `subsystem`) fall back to package-level defaults via `cmp.Or`. `s3ConfigFromProto` resolves `accessKeyId`, `secretAccessKey`, and `sessionToken` through a `ProjectionProvider` built from the `S3SecretProjection` in the proto. The helper `unsetToEmpty` absorbs `ErrNotFound` and `ErrProjectionMissing` (treating absent credentials as empty strings to support IRSA/WIF fallback), while propagating any other error. `gcsConfigFromProto` is a stub pending GCS credentials support. Adds `Option` / `clientOptions` / `WithInfraConfig` to `client.go` to allow tests and callers to inject an `infrastructure.Config` directly. Covers the new code with `TestConfigFromProto` (nil/missing config, unknown backend, GCS defaults, S3 full and partial credential resolution) and `TestS3ConfigFromProto` (all fields, optional fields, `ErrNotFound`, `ErrProjectionMissing`, per-field unexpected errors, and multi-error joining). Issues: - gitlab-com/gl-infra/platform/runway/team#875 - gitlab-com/gl-infra/platform/runway/team#876 -
Florian Forster authored
Sets `Backend` to `BackendGCS` or `BackendS3` in `gcsConfigFromProto` and `s3ConfigFromProto` respectively. Previously the field was left at its zero value, causing `NewWithConfig` to return an error. Updates all affected golden structs in `binding_test.go` and `client_test.go`. Adds `TestNew` in `client_test.go` (external test package) covering three cases: a valid S3 config that returns a usable `Client` with the correct `Backend`, `Name`, and non-nil `Bucket`; a config with no `ObjectStore` block returning `ErrNotConfigured`; and an `ObjectStore` with no backend set returning `ErrUnknownObjectStore`. The happy-path case drives a real `New` call against an in-process S3 server via `s3test.New`, exercising the full `configFromProto` → `s3ConfigFromProto` → `NewWithConfig` path including the `PrefixProvider` and `ProjectionProvider` secret chain.
-
Florian Forster authored
-
Florian Forster authored
Stops defaulting the name, namespace, and subsystem fields in `configFromProto`, leaving them as zero values when not explicitly set so `NewWithConfig` fills them in. Previously the defaults were applied via `cmp.Or` here as well, duplicating the default identity values across `binding.go` and `client.go`. Moves the `defaultName`, `defaultNamespace`, and `defaultSubsystem` consts to `client.go` and uses them in place of the inline string literals in `NewWithConfig`, so the defaults now live in a single place.
-
Florian Forster authored
Adds an `ErrMissingSecrets` sentinel and guards `configFromProto` against a nil `infraCfg.Secrets`, returning it before wrapping the provider. Without the check, a missing secret provider surfaced later as an opaque failure rather than a clear, matchable error. Corrects the `New` doc comment: `secret.ErrProjectionMissing` is returned when there is a user configuration issue – `ErrMissingSecrets` is only returned when there is an internal logic issue.
-
Luke Hollinda authored
feat(objectstore): Add proto-driven `New` constructor and `configFromProto` binding. See merge request !556 Merged-by:
Luke Hollinda <lhollinda@gitlab.com> Approved-by:
Luke Hollinda <lhollinda@gitlab.com> Reviewed-by:
Florian Forster <fforster@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Luke Hollinda authored
`secret.ErrProjectionMissing` errors are swallowed in `v2/objectstore/binding.go`'s `unsetToEmpty`
-
Luke Hollinda authored
The `--fix` flag was passed to every lint run, causing golangci-lint to exit 0, even when it rewrote files. This silently swallows lint failures at CI time. This commit removes the `--fix` flag and adds the ability to forward arbitrary flags to the script / mise task.
-
Luke Hollinda authored
golangci-lint error: objectstore/binding.go:20:59: QF1008: could remove embedded field "Config" from selector (staticcheck) if infraCfg == nil || infraCfg.Config == nil || infraCfg.Config.GetObjectStore() == nil { ^ objectstore/binding.go:32:18: QF1008: could remove embedded field "Config" from selector (staticcheck) os = infraCfg.Config.GetObjectStore() -
Florian Forster authored
Replaces the single `wantErr` field with `wantNewErr` and `wantDSNErr` so each case asserts the error against the specific stage that produces it, rather than matching the same expected error against both `newPSQLFormat` and `DSN`. Cases are updated to set whichever field applies. Switches the assertions to `require.ErrorIs`, dropping the manual `errors.Is` checks and the now-unused `errors` import. Marks the test parallel with `t.Parallel()`.
-
Florian Forster authored
-
Florian Forster authored
Introduces named constants for the `PostgreSQLProjection` JSON field names (`fieldHost`, `fieldPort`, and so on) and uses them throughout `DSN`, `resolveHost`, and `resolvePort` in place of scattered string literals, so the field names live in one place. Replaces the `hostArgs` and `portArgs` structs with plain parameters on `resolveHost` and `resolvePort`, passing the pool and non-pool config values directly. The secret-field arguments are no longer threaded through the callers since each helper now references the field constants itself. Simplifies the projection lookup in `newProjectionFormat` to use `o.infra.GetPostgresql()` instead of `o.infra.Config.GetPostgresql()`.
-
Florian Forster authored
Builds the secret resolution chain in `newAuthnFormat` by wrapping `infra.Secrets` in a `PrefixProvider` (mapping lookups under `redis/`) and a `ProjectionProvider`, stored on `authenticatedFormat` as a `secret.Provider`. `resolveSecret` now delegates to this provider keyed by JSON field name, and the per-field nil-projection plumbing in `AuthConfig` and `Addrs` is replaced by `resolveConfigKey`/`resolveSecret` calls against the new field-name constants (`fieldAddrs`, `fieldUsername`, and so on). `resolveConfigKey` lets an explicit value in the Redis config take precedence over the projected secret. Moves validation up front: `newAuthnFormat` now resolves the projection and rejects a nil secret provider with `ErrMissingSecrets` at construction time, and wraps the nil-infra case with `infrastructure.ErrNotConfigured`. The `projection` method becomes a package function taking the config directly, since resolution no longer hangs off `authenticatedFormat`. Adds `ignoreProjectionMissing` to strip `secret.ErrProjectionMissing` from the joined error, so optional projection fields that are unset no longer fail `AuthConfig`. The method now returns a zeroed `authConfig` on any remaining error rather than a partially populated one. Updates the tests to construct formats via `newAuthnFormat`, assert error sentinels with `errors.Is` and types with `reflect.Type`, and compare configs with `cmp.Diff`, adding `github.com/google/go-cmp`. Constructor-time failure cases (nil secrets, unknown format, missing projection) move to `Test_newAuthnFormat`, and a case covering config-over-secret precedence is added.
-
Florian Forster authored
Drops the redundant `cfg.GetUsername()` and `cfg.GetSentinelUsername()` seeding in `AuthConfig`, since `resolveConfigKey` already applies the config value with precedence, and declares `auth` and `errs` together. Rewrites `ignoreProjectionMissing` as an explicit loop that joins the non-`ErrProjectionMissing` errors, replacing the in-place `slices.DeleteFunc` and dropping the `slices` import. Lowercases the `secrets.Get` error string prefix to follow Go's convention that error strings are not capitalized.
-
Elliot Forbes authored
Make golangci-lint --fix opt-in to avoid swallowing errors in CI See merge request !559 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Luke Hollinda <lhollinda@gitlab.com>
-
Elliot Forbes authored
chore(v2/objectstore): remove misnomer objectstore.New() comment See merge request !558 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Luke Hollinda <lhollinda@gitlab.com>
-
Elliot Forbes authored
refactor(postgres): Wire `projectionFormat` through `ProjectionProvider` and `PrefixProvider`. See merge request !555 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by:
Florian Forster <fforster@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Elliot Forbes authored
refactor(redis): Migrate to `ProjectionProvider` and `PrefixProvider`. See merge request !560 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by:
Florian Forster <fforster@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Florian Forster <fforster@gitlab.com> -
Luke Hollinda authored
Named paramaters are standard here.
-
Luke Hollinda authored
More efficient, but primarily adjusted to silences gopls' diagnostic warning.
-
Luke Hollinda authored
Enables the function to be shared by both formats, removing noSecretFormat's duplicated nil-checks.
-
Luke Hollinda authored
The .go-build-cache job pointed GOMODCACHE/GOCACHE at paths outside $CI_PROJECT_DIR, which the runner silently skips, so every Go job ran cold. Relocate both caches into $CI_PROJECT_DIR and split them: - Modules: one shared, read-only cache seeded by a warm-go-mod-cache job (new prepare stage) so consumers can't overwrite the complete set with a partial one. golangci_lint depends on it to wait out the prepare stage. - Compiled objects: per-job cache keyed on $CI_JOB_NAME_SLUG, since output is specific to build tags, Go version, and -race.
-
Elliot Forbes authored
Post review follow-ups for fforster/redis See merge request !557 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Florian Forster <fforster@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Co-authored-by: Luke Hollinda <lhollinda@gitlab.com>
-
Elliot Forbes authored
fix(ci): make .go-build-cache actually cache Go artifacts See merge request !562 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by: GitLab Duo <gitlab-duo@gitlab.com> Co-authored-by:
Luke Hollinda <lhollinda@gitlab.com>