Improve cookiecutters: fix whitespace and fully implementing REST/GraphQL stream

Closes #119 (closed) and #116 (closed).

This update splits the streams.py file into multiple files:

  • client.py - this contains the base Stream classes, handling the bulk of the client implementation. Seeded by 3 conditionally-named files, each mutually exclusive from the others:
    • REST base streams: {%if "REST" == cookiecutter.stream_type %}client.py{%endif%}
    • GraphQL base streams: {%if "GraphQL" == cookiecutter.stream_type %}client.py{%endif%}
    • Custom base streams: {%if cookiecutter.stream_type not in ("REST", "GraphQL")%}client.py{%endif%}
  • streams.py - for SDK-based taps in the wild, column definitions and endpoint declarations make up the largest number of lines of code for. Breaking out this code from core API logic makes everything easier to read, and the streams.py file now is more declarative and easier to review.
  • auth.py - moving authenticators into a dedicated file makes the other cookiecutter files easier to develop and iterate on.

Following guidance from #116 (closed), this update also drastically improves the whitespace and newlines in the final output.

The REST and GraphQL client.py files now also have coverage for all functionality documented in docs/reference.md.

Edited by AJ Steers

Merge request reports

Loading