Commits on Source 6

  • Tristan Van Berkom's avatar
    source-determinism.py integration test: Dont use integration_cache fixture · 4ca37dcd
    Tristan Van Berkom authored
    This was just deadcode, it's not really used here.
    4ca37dcd
  • Tristan Van Berkom's avatar
    conftest.py: Use different artifact directory for integration tests · 6286d820
    Tristan Van Berkom authored
    To ensure we can run integration tests in parallel, use a tempdir
    for the artifact cache of each separate integration test run.
    
    This makes it possible to run multiple full test runs including
    integration tests in parallel under detox.
    6286d820
  • Tristan Van Berkom's avatar
    Merge branch 'tristan/detoxing-integration-tests' into 'master' · 471af316
    Tristan Van Berkom authored
    Make integration tests parallelizable
    
    See merge request !1077
    471af316
  • Tristan Van Berkom's avatar
    _scheduler: Refactor of queues and resources. · 69482602
    Tristan Van Berkom authored
    This branch makes the following changes:
    
      * jobs/job.py: No longer stores any interested resource list
    
        Jobs are ephemeral again, they only ever exist while they
        are running.
    
      * queues/queue.py: Revert to only handling lists of elements
    
        Elements pass through the queues, Queue.harvest_jobs()
        replaces Queue.pop_ready_jobs() and now the Queue stops
        creating jobs as soon as there are not enough resources
        for the job.
    
        Also removed unused `prepare()` abstract method.
    
      * queues/buildqueue.py: Adapt the part where we launch a job
    
        This part needs to be reworked anyway, just touch it up for
        now so that it doesnt break with the surrounding changes.
    
      * jobs/{cachesize,cleanup}job.py: Expose uniform complete callback
    
        Allows the scheduler to manage resource deallocation for these
        two job completions as a custom thing, at the same phase
        that the Queues take care of their own resource deallocation.
    
      * resources.py: No longer has knowledge of the job
    
        Since jobs are ephemeral, they are not a suitable place
        to store the resource identifiers, these must be provided
        by the callers wherever needed.
    
        Now the main Resources object is owned by the Scheduler
        but shared with Queues, each take care of managing the
        resources of the jobs they create through the same
        resource API.
    
      * scheduler.py: Reverted to only creating jobs on demand
    
        This changes the flow of the scheduler such that whenever
        jobs complete, the queues are interrogated for as many
        jobs which can run at the moment but not more; and this
        completely removes the waiting list.
    
        For the internal cache management jobs, we handle this
        with a little state instead of having a waiting list
        and only launch when the resources permit it.
    
    By abolishing the scheduler waiting list and creating jobs
    on demand, we fix the order of element processing and consequently
    fix issue #712.
    69482602
  • Tristan Van Berkom's avatar
    tests/frontend/order.py: Enable the test for build and fix the fetch tests · fea8030e
    Tristan Van Berkom authored
    With the scheduler changes, fetch jobs get automatically skipped
    so the output is changed, using a separate repo for each element
    fixes the test such that every fetch job gets a job launched.
    fea8030e
  • Tristan Van Berkom's avatar
    _artifactcache/artifactcache.py: Rephrase failure message · c70434c6
    Tristan Van Berkom authored
    It was saying "There is not enough space to build the given element.",
    this makes me think the error is associated to a specific element, but
    this does not make sense to show up in a cleanup task.
    
    Instead say "There is not enough space to complete the build.", which
    should be more clear that even after cleaning up there is not enough
    space.
    c70434c6
Loading
Loading