Hub SSOT - How MeltanoHub stores vs renders the API payloads

The current V1 hub API just basically serializes the raw taps and target metadata files as 1 large json and serves it up.

For the V2 endpoint depending on our implementation strategy we might find it better to decouple the file format and organization of the plugin definitions from the way that we serve the data. The hub files serve multiple purposes that might conflict: API data source, static site data source, organized for easy community maintenance and contribution.

For example (just for illustrating my point) we might find that for easier maintenance and contribution we want to split the files into directories and maybe theres a top level definition that is shared by all variants and each variant definition is its own file, later compiled by the API file generator to meet the API payload schema we want.

extractors/
    tap-csv/
        base.yml
        meltanolabs/
            variant.yml
        meltano/
            variant.yml
    tap-postgres/
        ...

The point is that we have the option to decouple the files from the API payload, and each have different goals (maintenance/contribution vs serving the front end static site vs API for serving an application efficiently).