Hub SSOT - File Bundle Versioning

Currently file bundles are installed based on matching naming conventions. Moving forward we'd like it to be more explicit and handle compatible versions better. As outlined by @aaronsteers in !226 (comment 898114534).

Currently:

  • Any file bundle that matches a plugin name exactly gets installed automatically along with that plugin.
  • File bundles can also be installed manually/explicitly.

In the future:

  1. Probably we should create a direct link between the plugin definition and optionally one or more files bundles that it should be installed with.
  2. In the case of dbt, for example, this probably is declared using one of the following:
    1. the main dbt plugin declares a dependency on the files-dbt repo. Perhaps not ideal because there's no 'plugin definition' in this case.
    2. the main dbt plugin lists plugin dependencies (including files bundles) using their keys: { 'type': 'files', 'name': 'dbt', 'variant': 'meltano'}. Basically this is a list of the primary keys needed to find the ref in the index.
    3. then main dbt plugin has something like {'dependencies': 'files': ['name': 'dbt', 'variant': 'meltano']}. This is parallel to how we define plugins in discovery.yml today, only more sparse. The plugin type is 'files' in this example but in theory it could be any plugin type(s).
  3. The declaration method doesn't have to dictate how this is exposed in the API. The API might render the above info simply as: {'dependencies': ['ref': 'https://'], ['ref': 'https://']} and point to one or more detail API endpoints.