Skip to content

Compliance with Spec, as documented in new meltano.com Singer Spec docs

With meltano!2083 (merged) now documenting the Singer Spec in an easier-to-digest fashion, let's apply that page as a test and checklist for SDK completeness.

  1. Metadata
    • We haven't invested much in discoverable metadata. We should declare these properties automatically when we can.
    • If and when the SDK cannot automatically implement the full set of discoverable metadata attributes, we should provide easy-to-use controls for the developer to do so. be sensitive in nature.
  2. Stream selection (TODO, as documented in #7 (closed) and resolving in !121 (merged))
    • Additionally, as related to the metadata topic above:
      • inclusion - Should default to automatic for replication keys and primary keys, available for all others.
      • selected-by-default - Should default to true
        • Should be overridable so the developer can use false when the stream is known to be sensitive in nature
  3. Add logging for metrics types:
    • http_request_duration (type: timer) - If personally find these messages overly verbose and I often wish I could mute them. Nevertheless, we should at least support emitting them.
    • record_count (type: counter) - Per the cadence of STATE messages, we can simultaneously emit counter metrics.

Documentation updates:

Once completed, the following documentation pages should also be updated:

Edited by AJ Steers