Loading
Commits on Source 23
-
Piotr Skorupa authored
-
Piotr Skorupa authored
This fixes two bugs with serializing Snowplow events, which resulted in bad events when testing end-to-end with Snowplow Micro: stm not "ms since epoch", se_va not BigDecimal. Bug 1: stm is never populated due to a range-copy bug in emitter.go:231-235 Event is a value copy of eventRows[i]. Assigning to event.SentTimestamp mutates the copy and is discarded; the slice is unchanged. The empty string then arrives at the collector as stm="", which fails the atomic-schema parse to "ms since epoch". Bug 2: structured-event fields are sent as empty strings on ue events in event.go:36-40 — the se_* fields have no omitempty, so they're always serialized. The atomic schema expects se_va to be a BigDecimal (i.e. a numeric string). Empty string fails parsing. For an unstructured event (e=ue), all se_* fields should be absent.
-
Piotr Skorupa authored
TestEmitter_DoSend_PopulatesSentTimestampOnAllEvents previously captured only the last request body and signalled on the first arrival, so when the emitter's send loop fired between the two TrackEvent calls the test inspected a single-event batch and failed "\"1\" is not greater than or equal to \"2\"" under -race on CI. Append every received event across all requests into a shared slice and wait until the cumulative count reaches the expected total. The assertion is now robust whether the emitter delivers all events in one batch or splits them across multiple POSTs, and the production fix in 4f8d16eb is still pinned because every observed event must carry a non-empty, ms-since-epoch stm. Co-Authored-By:
Claude Opus 4.7 (1M context) <noreply@anthropic.com>
-
João Pereira authored
The previous check only rejected an untyped nil interface, so a typed-nil value passed as `EventContext` (e.g. `var c *MyContext = nil` or a typed-nil `func`) would slip past the guard and panic on the subsequent `ctx.Schema()` / `ctx.Data()` call. Widen the guard to cover every nilable kind reachable through an interface (pointer, map, channel, func, slice, interface). `reflect.Value.IsNil` panics on non-nilable kinds, so the kind check stays first; value-typed contexts like `GitLabStandardContext` (Kind == Struct) fall through. Add a table-driven regression test exercising both pointer and func typed nils, and tighten the untyped-nil assertion to "is nil" so it can't accidentally pass against the typed-nil branch.
-
João Pereira authored
The previous `GitLabStandardContext.Validate` checked required Environment, the Realm/DeploymentType enums, and the UserID type. The gitlab_standard iglu schema 1-1-8 enforces more: every integer ID caps at 2^31-1 with a minimum of 0, and around twenty string fields have maxLength constraints in three tiers (32, 64, 255). Values that violate these constraints compile but get rejected by the Snowplow enricher at ingest time as bad events, making the failure invisible to the caller. Add three helpers covering the gaps: - `validateStringLengths` checks every string field against its tier. - `validateIntegerRanges` checks every integer ID and the elements of `FeatureEnabledByNamespaceIDs` against `[0, 2^31-1]`. - `validateUserID` checks string length and integer range depending on the dynamic type (the schema permits both). Error messages use lowercase field names to match Go convention and the project's custom instructions on error formatting. Test coverage: at-limit (valid) and over-limit (invalid) cases for each tier, integer boundaries (0, MaxInt32, MaxInt32+1), per-element index in the array field, and `uint`-platform-width coverage in the user_id bounds tests.
-
João Pereira authored
- Remove `GitLabStandardContext.AsEventContext` and its dedicated test. The method was a convenience wrapper around the no-op conversion to `EventContext`; since the type already satisfies the interface directly, the method added a permanent backward-compatibility obligation for no functional gain. - Convert the realm/deployment_type validation tests from `map[string]bool` to slice-of-anonymous-structs with a `name` field, matching the project's standard table-test pattern and producing deterministic subtest order. - Rename `TestTrackEvent_BackwardCompatible` to `TestTrackEvent_OmitsContextEncoded` so the test name describes the unique invariant it pins (no `cx` field) rather than describing what the broader `TestTrackEvent_Success` already covers. - Add `TestTrackEventWithContexts_AcceptsContextWithoutValidate` so the optional-`Validate` branch of `TrackEventWithContexts` is exercised in isolation (the existing tests always paired `fakeContext` with a `GitLabStandardContext`).
-
João Pereira authored
- Package doc: - Fix batch-size statement: emitter sends up to 100 events per batch (`sendingAmount = 100` in emitter.go), not 5. Two locations updated. - Make the `ArtifactRegistryEventContext` example self-contained by showing the minimal `Schema()` and `Data()` implementation a caller needs. Drop the orphan `ARNamespaceID` field that wasn't shown in the type definition. - Fix a pre-existing missing closing quote on `"global-user-789"`. - Update the Custom Contexts section to reflect the broader constraints `Validate` now enforces (string maxLengths and integer ranges). - `GitLabStandardContext` godoc: - Reword the pointer-fields note so it doesn't read as exhaustive (currently only `IsGitLabTeamMember`). - Replace the "set to string or int/int64 only" guidance for `UserID` with the actual accepted set (nil, string, every signed/unsigned integer width). - Drop the reference to the removed `AsEventContext` method and note that the type satisfies `EventContext` directly. - `stringMax*` constant comment rewritten to plural so it matches what the block actually declares. - `encodeContexts` godoc no longer claims it rejects empty input; callers are guarded upstream by `TrackEventWithContexts`. -
João Pereira authored
Define `type Realm string` and `type DeploymentType string` so the Realm and DeploymentType fields of `GitLabStandardContext` reject free-form string assignments at the call site. With the previous untyped string constants, `ctx.Realm = "anyhing"` compiled and only surfaced at runtime through `Validate` or at the Snowplow enricher. The runtime enum check in `Validate` is kept so explicit casts (`Realm("typo")`) and values coming from external sources are still caught. Wire format is unchanged: both types have an underlying `string`, so the JSON output is identical. -
João Pereira authored
Replace `UserID any` with a sealed-interface tagged union (`UserID`, `UserIDString`, `UserIDInt`) so the two variants the gitlab_standard schema permits are enforced at compile time. The previous `any` field let any concrete type through and relied entirely on a runtime type switch in `Validate` to reject invalid ones. The unexported `isUserID()` marker prevents callers from defining additional variants, so `validateUserID` reduces from ~40 lines of per-integer-width cases to a three-case switch. Wire format is preserved: `UserIDString` has underlying type `string` and marshals as a JSON string, `UserIDInt` has underlying type `int64` and marshals as a JSON number. A new `_UserID_JSONShape` test pins this so a future change to the underlying types or a custom `MarshalJSON` regression would be caught. Also document the schema-versioning policy on `GitLabStandardContext`: additive schema revisions update the type in place; breaking revisions introduce a new type (e.g. `GitLabStandardContextV2`) so existing callers continue to compile.
-
João Pereira authored
Pass over the package's doc comments to remove restating-the-obvious, collapse multi-paragraph explanations into single tighter paragraphs, and fix a factual error: Validate's previous docstring said the constraints are "enforced at compile time", but Validate runs at call time. No behavior change.
-
João Pereira authored
JSON Schema's `maxLength` is defined in unicode code points, but `len()` on a Go string returns bytes. Fields likely to contain non-ASCII text (`user_type`, `feature_category`, `model_name`, etc.) were falsely rejected when the byte count exceeded the limit but the code-point count did not. The Snowplow collector's iglu validator is the authority and would accept those payloads. Switch `validateStringLengths` and `validateUserID` to `utf8.RuneCountInString`. Add regression cases using `strings.Repeat("中", N)` to pin the behavior at the boundary for both code paths. -
João Pereira authored
The prior wording implied `Data()` itself is base64-encoded. In fact the returned value is wrapped in `{schema, data}`, placed in a contexts envelope, JSON-marshalled, and then base64-encoded as a whole. Spell that out so implementers don't think their payload must be base64-friendly, and add the JSON-marshalable contract. -
João Pereira authored
TrackEvent delegates to TrackEventWithContexts, which also returns marshal failures and emitter errors beyond ErrEmitterStopped. The prior wording ("eventName is empty or the tracker is stopped") read as exhaustive and could lead callers to handle only those two cases. Point at TrackEventWithContexts for the full contract. -
João Pereira authored
The trailing `return nil` after the type switch was unreachable today (UserID is sealed to nil/UserIDString/UserIDInt) but a future contributor adding a fourth variant without updating the switch would silently bypass validation. Replace the dead-code return with an explicit `default` returning an "unhandled variant" error so missed cases fail loudly.
-
João Pereira authored
The omitempty rationale listed `ue_px` and `se_*` but skipped `cx`, which is also omitempty and is shared by both event types (unstructured events with custom contexts and structured billing events both populate it). Adding the sentence so a maintainer doesn't remove omitempty under the assumption that cx is always populated.
-
João Pereira authored
The example defined ArtifactRegistryEventContext inline then instantiated it as myService.ArtifactRegistryEventContext{}, implying a separate package. Drop the prefix so the inline definition and usage are consistent. Also change the standard-context example's InstanceID value from "unique-instance-id" — easily confused with the separate UniqueInstanceID field — to a more representative placeholder. -
João Pereira authored
The iglu schema version (1-1-8) was repeated in six inline comments across this file. When the schema bumps to an additive revision (e.g. 1-1-9), those comments go stale silently — only the SchemaGitLabStandard constant in self_describing_json.go actually needs to change. Point every comment at the constant instead, so there is a single source of truth.
-
João Pereira authored
The production typed-nil guard in tracker.go switches on six nilable reflect.Kind values; the test only exercised two (pointer and func). Add named slice, map, and chan EventContext implementations so the remaining reachable kinds are pinned — a regression that drops any of these from the switch panics instead of erroring, and the test should catch that. (reflect.Interface is in the production case list but is not reachable for EventContext implementers via reflect.ValueOf, so no test is added for it.)
-
João Pereira authored
The pre-existing string-length test covered one representative field per tier (32, 64, 255). A regression that moved e.g. \`source\` to the long tier, or that dropped a field from the validateStringLengths table, would not be caught. Add an exhaustive table-driven test setting each of the 21 fields to tier + 1 runes and asserting Validate rejects it naming that field.
-
João Pereira authored
Validate compares Realm and DeploymentType values against the package constants themselves, so a typo like `RealmDedicated = "Dedicated"` would still pass Validate while breaking downstream Snowflake queries that read `realm = 'dedicated'`. Add tests that marshal a context for every constant and assert the JSON value matches the iglu schema's exact literal.
-
João Pereira authored
The encodeContexts marshal-failure branch returns a wrapped error when a context's Data() is not JSON-marshalable, and the caller in TrackEventWithContexts re-wraps it as "failed to encode custom contexts". Neither was exercised by an existing test, so a regression that dropped the error wrapping or that proceeded to enqueue the event despite a marshal failure would go unnoticed. Use a fakeContext whose Data() is a chan int (not marshalable) and assert the error message and that no event is queued.
-
João Pereira authored
The existing validation-failure and typed-nil tests pass a single context, so they would not catch a regression that hard-codes \`index 0\` or drops the schema URI from the error message. Add two multi-context tests: - A trailing failingContext: assert the error contains \`index 1\`, the offending schema URI, and the underlying validation error. - A trailing typed-nil pointer: assert the error contains \`index 1\` and \`typed-nil ptr\`.
-
Elliot Forbes authored
feat(v2/events/snowplow) add custom context support to v2 Snowplow See merge request !498 Merged-by:
Elliot Forbes <eforbes@gitlab.com>
Approved-by: Niko Belokolodov <nbelokolodov@gitlab.com> Approved-by:
Elliot Forbes <eforbes@gitlab.com>
Reviewed-by: João Pereira <jpereira@gitlab.com> Reviewed-by:
GitLab Duo <gitlab-duo@gitlab.com> Reviewed-by:
Elliot Forbes <eforbes@gitlab.com>
Co-authored-by: João Pereira <jpereira@gitlab.com> Co-authored-by:
Piotr Skorupa <pskorupa@gitlab.com>