Adjust Source / SourceFetcher API contract
As discussed in these comments:
And later on IRC, we need to make a subtle change the Source
/ SourceFetcher
API contract to ensure that all user provided configuration data will be consumed at Plugin.configure()
time so that it can be appropriately validated early on, before encountering errors in mid-flight.
We agreed that:
- The documentation will assert that a Source plugin must call either
Source.translate_url()
orSource.mark_download_url()
atPlugin.configure()
time, for every URL that the user has provided in the configuration - It will be documented that the order is significant
- The first call to
Source.mark_download_url()
duringPlugin.configure()
will be the one used to derive the primary alias, which will be employed atSource.track()
time.
- The first call to
- It will be asserted later on, after instantiating each
SourceFetcher()
but before fetching- We will raise an assertion if a URL marked by the
SourceFetcher()
has an alias, and has never been seen duringPlugin.configure()
, this will issue aBUG
causing, the plugin author to fix their code.
- We will raise an assertion if a URL marked by the