Skip to content
  • Tristan Van Berkom's avatar
    _stream.py, _pipeline.py: Refactoring of the pipeline itself · 5bdc0a79
    Tristan Van Berkom authored
    Here the pipeline becomes essentially stateless, some dangling
    state remains to be factored out because of frontend accesses
    which will be changed in a later commit.
    
    Essentially, the Pipeline.load() method no longer has any knowledge
    of the specific purposes of the loaded targets, and now takes
    a list of target groups and returns a corresponding list of element
    groups.
    
    The Stream() business logic methods now use other pipeline helper
    methods to create and filter lists from the loaded target elements.
    
    The Stream() also finally absorbs the Scheduler frontend facing
    APIs. However Queues are still exposed on the Stream object for
    logging purposes and through callbacks such that the frontend can
    retry elements.
    5bdc0a79