Skip to content

Add Local Repo Benchmarking Testing and other corrections

Lachlan requested to merge lachlanmackenzie/LocalRepoBenchmarkingTesting-rebase into master

Description

Currently it is envisaged that benchmarking will be carried out locally (at desk) by developers and as such it is possible that a separate repository for local work will be created. As part of the process of gaining acceptance for this approach a development branch was created for evaluation purposes, this branch was extended to correct a number of long standing issues in the current repository. This MR will merge these changes (together with the local orchestration work) as an interim position.

Changes proposed in this merge request:

  • Add orchestration scripts allowing benchmarking using local repositories.
  • Add caching of repositories so that multiple clones from remote sources are not required in each job.
  • Reference each test result by version and source.
  • Add standard deviation as a metric for evaluation
  • Add script to digest results from raw json files.
  • Correct numerous errors in graphing and docker container/image/volume handling.

This merge request, when merged, will address issue/bug:

As per description.

Merge request reports