Compliance with Spec, as documented in new meltano.com Singer Spec docs
With meltano!2083 (merged) now documenting the Singer Spec in an easier-to-digest fashion, let's apply that page as a test and checklist for SDK completeness.
-
Metadata - We haven't invested much in
discoverable metadata
. We should declare these properties automatically when we can. - If and when the SDK cannot automatically implement the full set of discoverable metadata attributes, we should provide easy-to-use controls for the developer to do so. be sensitive in nature.
- We haven't invested much in
-
Stream selection (TODO, as documented in #7 (closed) and resolving in !121 (merged)) - Additionally, as related to the
metadata
topic above:-
inclusion
- Should default toautomatic
for replication keys and primary keys,available
for all others. -
selected-by-default
- Should default totrue
-
Should be overridable so the developer can use false
when the stream is known to be sensitive in nature
-
-
- Additionally, as related to the
-
Add logging for metrics types: -
http_request_duration
(type:timer
) - If personally find these messages overly verbose and I often wish I could mute them. Nevertheless, we should at least support emitting them. -
record_count
(type:counter
) - Per the cadence of STATE messages, we can simultaneously emit counter metrics.
-
Documentation updates:
Once completed, the following documentation pages should also be updated:
-
docs/implementation/catalog_metatadata.md
: https://sdk.meltano.com/en/latest/implementation/catalog_metadata.html -
docs/implementation/metrics.md
-
docs/implementation/catalog.md
: https://sdk.meltano.com/en/latest/implementation/cli.html#input-catalog-catalog
Edited by AJ Steers