Commits on Source 23

  • Piotr Skorupa's avatar
  • Piotr Skorupa's avatar
    fix(v2/events/snowplow): fix atomic_field_parse_error validation errors with Snowplow Micro · eed96c0a
    Piotr Skorupa authored
    This fixes two bugs with serializing Snowplow events, which resulted in
    bad events when testing end-to-end with Snowplow Micro: stm not "ms
    since epoch", se_va not BigDecimal.
    
    Bug 1: stm is never populated due to a range-copy bug in
    emitter.go:231-235
    
    Event is a value copy of eventRows[i]. Assigning to event.SentTimestamp
    mutates the copy and is discarded; the slice is unchanged. The empty
    string then arrives at the collector as stm="", which fails the
    atomic-schema parse to "ms since epoch".
    
    Bug 2: structured-event fields are sent as empty strings on ue events
    in event.go:36-40 — the se_* fields have no omitempty, so they're
    always serialized.
    
    The atomic schema expects se_va to be a BigDecimal (i.e. a numeric
    string). Empty string fails parsing. For an unstructured event (e=ue),
    all se_* fields should be absent.
    eed96c0a
  • Piotr Skorupa's avatar
    test(v2/events/snowplow): make stm regression test robust to batching · deeacc46
    Piotr Skorupa authored
    TestEmitter_DoSend_PopulatesSentTimestampOnAllEvents previously captured
    only the last request body and signalled on the first arrival, so when
    the emitter's send loop fired between the two TrackEvent calls the test
    inspected a single-event batch and failed
    "\"1\" is not greater than or equal to \"2\"" under -race on CI.
    
    Append every received event across all requests into a shared slice and
    wait until the cumulative count reaches the expected total. The assertion
    is now robust whether the emitter delivers all events in one batch or
    splits them across multiple POSTs, and the production fix in 4f8d16eb is
    still pinned because every observed event must carry a non-empty,
    ms-since-epoch stm.
    
    Co-Authored-By: default avatarClaude Opus 4.7 (1M context) <noreply@anthropic.com>
    deeacc46
  • João Pereira's avatar
    fix(v2/events/snowplow): guard against typed-nil contexts of all nilable kinds · 7b13d667
    João Pereira authored
    The previous check only rejected an untyped nil interface, so a typed-nil
    value passed as `EventContext` (e.g. `var c *MyContext = nil` or a typed-nil
    `func`) would slip past the guard and panic on the subsequent
    `ctx.Schema()` / `ctx.Data()` call.
    
    Widen the guard to cover every nilable kind reachable through an interface
    (pointer, map, channel, func, slice, interface). `reflect.Value.IsNil`
    panics on non-nilable kinds, so the kind check stays first; value-typed
    contexts like `GitLabStandardContext` (Kind == Struct) fall through.
    
    Add a table-driven regression test exercising both pointer and func typed
    nils, and tighten the untyped-nil assertion to "is nil" so it can't
    accidentally pass against the typed-nil branch.
    7b13d667
  • João Pereira's avatar
    feat(v2/events/snowplow): enforce iglu schema lengths and integer ranges in Validate · f2039a57
    João Pereira authored
    The previous `GitLabStandardContext.Validate` checked required Environment,
    the Realm/DeploymentType enums, and the UserID type. The gitlab_standard
    iglu schema 1-1-8 enforces more: every integer ID caps at 2^31-1 with a
    minimum of 0, and around twenty string fields have maxLength constraints
    in three tiers (32, 64, 255). Values that violate these constraints
    compile but get rejected by the Snowplow enricher at ingest time as bad
    events, making the failure invisible to the caller.
    
    Add three helpers covering the gaps:
    
    - `validateStringLengths` checks every string field against its tier.
    - `validateIntegerRanges` checks every integer ID and the elements of
      `FeatureEnabledByNamespaceIDs` against `[0, 2^31-1]`.
    - `validateUserID` checks string length and integer range depending on
      the dynamic type (the schema permits both).
    
    Error messages use lowercase field names to match Go convention and the
    project's custom instructions on error formatting.
    
    Test coverage: at-limit (valid) and over-limit (invalid) cases for each
    tier, integer boundaries (0, MaxInt32, MaxInt32+1), per-element index in
    the array field, and `uint`-platform-width coverage in the user_id
    bounds tests.
    f2039a57
  • João Pereira's avatar
    refactor(v2/events/snowplow): apply review feedback on custom context API · eeae2500
    João Pereira authored
    - Remove `GitLabStandardContext.AsEventContext` and its dedicated test.
      The method was a convenience wrapper around the no-op conversion to
      `EventContext`; since the type already satisfies the interface
      directly, the method added a permanent backward-compatibility
      obligation for no functional gain.
    - Convert the realm/deployment_type validation tests from
      `map[string]bool` to slice-of-anonymous-structs with a `name` field,
      matching the project's standard table-test pattern and producing
      deterministic subtest order.
    - Rename `TestTrackEvent_BackwardCompatible` to
      `TestTrackEvent_OmitsContextEncoded` so the test name describes the
      unique invariant it pins (no `cx` field) rather than describing what
      the broader `TestTrackEvent_Success` already covers.
    - Add `TestTrackEventWithContexts_AcceptsContextWithoutValidate` so the
      optional-`Validate` branch of `TrackEventWithContexts` is exercised
      in isolation (the existing tests always paired
      `fakeContext` with a `GitLabStandardContext`).
    eeae2500
  • João Pereira's avatar
    docs(v2/events/snowplow): correct package and inline documentation · eb0fffaf
    João Pereira authored
    - Package doc:
      - Fix batch-size statement: emitter sends up to 100 events per batch
        (`sendingAmount = 100` in emitter.go), not 5. Two locations updated.
      - Make the `ArtifactRegistryEventContext` example self-contained by
        showing the minimal `Schema()` and `Data()` implementation a caller
        needs. Drop the orphan `ARNamespaceID` field that wasn't shown in
        the type definition.
      - Fix a pre-existing missing closing quote on `"global-user-789"`.
      - Update the Custom Contexts section to reflect the broader
        constraints `Validate` now enforces (string maxLengths and integer
        ranges).
    - `GitLabStandardContext` godoc:
      - Reword the pointer-fields note so it doesn't read as exhaustive
        (currently only `IsGitLabTeamMember`).
      - Replace the "set to string or int/int64 only" guidance for `UserID`
        with the actual accepted set (nil, string, every signed/unsigned
        integer width).
      - Drop the reference to the removed `AsEventContext` method and
        note that the type satisfies `EventContext` directly.
    - `stringMax*` constant comment rewritten to plural so it matches what
      the block actually declares.
    - `encodeContexts` godoc no longer claims it rejects empty input;
      callers are guarded upstream by `TrackEventWithContexts`.
    eb0fffaf
  • João Pereira's avatar
    feat(v2/events/snowplow): type Realm and DeploymentType as named string types · 6100362c
    João Pereira authored
    Define `type Realm string` and `type DeploymentType string` so the Realm
    and DeploymentType fields of `GitLabStandardContext` reject free-form
    string assignments at the call site. With the previous untyped string
    constants, `ctx.Realm = "anyhing"` compiled and only surfaced at runtime
    through `Validate` or at the Snowplow enricher.
    
    The runtime enum check in `Validate` is kept so explicit casts
    (`Realm("typo")`) and values coming from external sources are still
    caught.
    
    Wire format is unchanged: both types have an underlying `string`, so the
    JSON output is identical.
    6100362c
  • João Pereira's avatar
    feat(v2/events/snowplow): seal UserID as a tagged union and document versioning · 797115fc
    João Pereira authored
    Replace `UserID any` with a sealed-interface tagged union (`UserID`,
    `UserIDString`, `UserIDInt`) so the two variants the gitlab_standard
    schema permits are enforced at compile time. The previous `any` field
    let any concrete type through and relied entirely on a runtime type
    switch in `Validate` to reject invalid ones.
    
    The unexported `isUserID()` marker prevents callers from defining
    additional variants, so `validateUserID` reduces from ~40 lines of
    per-integer-width cases to a three-case switch.
    
    Wire format is preserved: `UserIDString` has underlying type `string`
    and marshals as a JSON string, `UserIDInt` has underlying type `int64`
    and marshals as a JSON number. A new `_UserID_JSONShape` test pins
    this so a future change to the underlying types or a custom
    `MarshalJSON` regression would be caught.
    
    Also document the schema-versioning policy on `GitLabStandardContext`:
    additive schema revisions update the type in place; breaking revisions
    introduce a new type (e.g. `GitLabStandardContextV2`) so existing
    callers continue to compile.
    797115fc
  • João Pereira's avatar
    docs(v2/events/snowplow): trim verbose comments · 9ba32bad
    João Pereira authored
    Pass over the package's doc comments to remove restating-the-obvious,
    collapse multi-paragraph explanations into single tighter paragraphs, and
    fix a factual error: Validate's previous docstring said the constraints are
    "enforced at compile time", but Validate runs at call time.
    
    No behavior change.
    9ba32bad
  • João Pereira's avatar
    fix(v2/events/snowplow): count code points, not bytes, in maxLength validation · ad68b014
    João Pereira authored
    JSON Schema's `maxLength` is defined in unicode code points, but `len()`
    on a Go string returns bytes. Fields likely to contain non-ASCII text
    (`user_type`, `feature_category`, `model_name`, etc.) were falsely
    rejected when the byte count exceeded the limit but the code-point count
    did not. The Snowplow collector's iglu validator is the authority and
    would accept those payloads.
    
    Switch `validateStringLengths` and `validateUserID` to
    `utf8.RuneCountInString`. Add regression cases using `strings.Repeat("中", N)`
    to pin the behavior at the boundary for both code paths.
    ad68b014
  • João Pereira's avatar
    docs(v2/events/snowplow): clarify EventContext.Data encoding chain · 46e6a2f7
    João Pereira authored
    The prior wording implied `Data()` itself is base64-encoded. In fact
    the returned value is wrapped in `{schema, data}`, placed in a contexts
    envelope, JSON-marshalled, and then base64-encoded as a whole. Spell
    that out so implementers don't think their payload must be
    base64-friendly, and add the JSON-marshalable contract.
    46e6a2f7
  • João Pereira's avatar
    docs(v2/events/snowplow): drop closed error enumeration from TrackEvent godoc · de416b94
    João Pereira authored
    TrackEvent delegates to TrackEventWithContexts, which also returns
    marshal failures and emitter errors beyond ErrEmitterStopped. The
    prior wording ("eventName is empty or the tracker is stopped") read
    as exhaustive and could lead callers to handle only those two cases.
    Point at TrackEventWithContexts for the full contract.
    de416b94
  • João Pereira's avatar
    refactor(v2/events/snowplow): make validateUserID switch exhaustive via default · f2439ef9
    João Pereira authored
    The trailing `return nil` after the type switch was unreachable today
    (UserID is sealed to nil/UserIDString/UserIDInt) but a future
    contributor adding a fourth variant without updating the switch would
    silently bypass validation. Replace the dead-code return with an
    explicit `default` returning an "unhandled variant" error so missed
    cases fail loudly.
    f2439ef9
  • João Pereira's avatar
    docs(v2/events/snowplow): note cx is shared and omitempty in Event godoc · 19127d5f
    João Pereira authored
    The omitempty rationale listed `ue_px` and `se_*` but skipped `cx`,
    which is also omitempty and is shared by both event types
    (unstructured events with custom contexts and structured billing
    events both populate it). Adding the sentence so a maintainer doesn't
    remove omitempty under the assumption that cx is always populated.
    19127d5f
  • João Pereira's avatar
    docs(v2/events/snowplow): clean up custom-context example in package doc · 54561ec9
    João Pereira authored
    The example defined ArtifactRegistryEventContext inline then
    instantiated it as myService.ArtifactRegistryEventContext{}, implying
    a separate package. Drop the prefix so the inline definition and
    usage are consistent.
    
    Also change the standard-context example's InstanceID value from
    "unique-instance-id" — easily confused with the separate
    UniqueInstanceID field — to a more representative placeholder.
    54561ec9
  • João Pereira's avatar
    docs(v2/events/snowplow): reference SchemaGitLabStandard for schema version · b3ac937d
    João Pereira authored
    The iglu schema version (1-1-8) was repeated in six inline comments
    across this file. When the schema bumps to an additive revision
    (e.g. 1-1-9), those comments go stale silently — only the
    SchemaGitLabStandard constant in self_describing_json.go actually
    needs to change.
    
    Point every comment at the constant instead, so there is a single
    source of truth.
    b3ac937d
  • João Pereira's avatar
    test(v2/events/snowplow): cover slice, map, and chan in typed-nil guard · ddf1e4aa
    João Pereira authored
    The production typed-nil guard in tracker.go switches on six nilable
    reflect.Kind values; the test only exercised two (pointer and func).
    Add named slice, map, and chan EventContext implementations so the
    remaining reachable kinds are pinned — a regression that drops any
    of these from the switch panics instead of erroring, and the test
    should catch that.
    
    (reflect.Interface is in the production case list but is not reachable
    for EventContext implementers via reflect.ValueOf, so no test is added
    for it.)
    ddf1e4aa
  • João Pereira's avatar
    test(v2/events/snowplow): pin every string field's maxLength tier · f0070cbd
    João Pereira authored
    The pre-existing string-length test covered one representative field
    per tier (32, 64, 255). A regression that moved e.g. \`source\` to the
    long tier, or that dropped a field from the validateStringLengths
    table, would not be caught.
    
    Add an exhaustive table-driven test setting each of the 21 fields to
    tier + 1 runes and asserting Validate rejects it naming that field.
    f0070cbd
  • João Pereira's avatar
    test(v2/events/snowplow): pin Realm and DeploymentType wire values · 480c947a
    João Pereira authored
    Validate compares Realm and DeploymentType values against the package
    constants themselves, so a typo like `RealmDedicated = "Dedicated"`
    would still pass Validate while breaking downstream Snowflake queries
    that read `realm = 'dedicated'`. Add tests that marshal a context for
    every constant and assert the JSON value matches the iglu schema's
    exact literal.
    480c947a
  • João Pereira's avatar
    test(v2/events/snowplow): cover encodeContexts marshal error path · 6e88afa6
    João Pereira authored
    The encodeContexts marshal-failure branch returns a wrapped error
    when a context's Data() is not JSON-marshalable, and the caller in
    TrackEventWithContexts re-wraps it as "failed to encode custom
    contexts". Neither was exercised by an existing test, so a regression
    that dropped the error wrapping or that proceeded to enqueue the
    event despite a marshal failure would go unnoticed.
    
    Use a fakeContext whose Data() is a chan int (not marshalable) and
    assert the error message and that no event is queued.
    6e88afa6
  • João Pereira's avatar
    test(v2/events/snowplow): pin error index and schema for multi-context cases · c75635a6
    João Pereira authored
    The existing validation-failure and typed-nil tests pass a single
    context, so they would not catch a regression that hard-codes
    \`index 0\` or drops the schema URI from the error message.
    
    Add two multi-context tests:
    - A trailing failingContext: assert the error contains \`index 1\`,
      the offending schema URI, and the underlying validation error.
    - A trailing typed-nil pointer: assert the error contains
      \`index 1\` and \`typed-nil ptr\`.
    c75635a6
  • Elliot Forbes's avatar
    Merge branch 'psk-add-custom-context-support-to-v2' into 'master' · 705642e1
    Elliot Forbes authored
    feat(v2/events/snowplow) add custom context support to v2 Snowplow
    
    See merge request !498
    
    Merged-by: Elliot Forbes's avatarElliot Forbes <eforbes@gitlab.com>
    Approved-by: default avatarNiko Belokolodov <nbelokolodov@gitlab.com>
    Approved-by: Elliot Forbes's avatarElliot Forbes <eforbes@gitlab.com>
    Reviewed-by: default avatarJoão Pereira <jpereira@gitlab.com>
    Reviewed-by: default avatarGitLab Duo <gitlab-duo@gitlab.com>
    Reviewed-by: Elliot Forbes's avatarElliot Forbes <eforbes@gitlab.com>
    Co-authored-by: default avatarJoão Pereira <jpereira@gitlab.com>
    Co-authored-by: default avatarPiotr Skorupa <pskorupa@gitlab.com>
    705642e1
Loading
Loading