Skip to content

Add complete failures to results score

Grant Young requested to merge gy-score-update into master

No completed iterations tests should still be counted instead of ignored

          /\      |‾‾|  /‾‾/  /‾/
     /\  /  \     |  |_/  /  / /
    /  \/    \    |      |  /  ‾‾\
   /          \   |  |‾\  \ | (_) |
  / __________ \  |__|  \__\ \___/ .io

  execution: local--------------------------------------------------]   servertor
     output: -
     script: k6/tests/api/api_v4_groups_group.js

    duration: -, iterations: -
         vus: 1, max: 2

time="2020-04-30T16:52:46+01:00" level=info-------------------------] starting
time="2020-04-30T16:52:46+01:00" level=info msg="RPS Threshold: 1.60/s (8)"
time="2020-04-30T16:52:46+01:00" level=info msg="TTFB P90 Threshold: 1500ms"
time="2020-04-30T16:52:46+01:00" level=info msg="Success Rate Threshold: 95%"
time="2020-04-30T16:52:47+01:00" level=info msg=Running i=0 t=985.032963ms
time="2020-04-30T16:52:48+01:00" level=info msg=Running i=0 t=1.985245434s
time="2020-04-30T16:52:49+01:00" level=info msg=Running i=0 t=2.98506031s
time="2020-04-30T16:52:50+01:00" level=info msg=Running i=0 t=3.985061744s
time="2020-04-30T16:52:51+01:00" level=info msg=Running i=0 t=4.985238336s
time="2020-04-30T16:52:51+01:00" level=info msg="Test finished" i=0 t=5.000106698s
time="2020-04-30T16:52:51+01:00" level=warning msg="No data generated, because no script iterations finished, consider making the test duration longer"

    █ API - Group Details

    data_received........: 0 B 0 B/s
    data_sent............: 0 B 0 B/s
    iteration_duration...: avg=0.14ms min=0.14ms med=0.14ms max=0.14ms p(90)=0.14ms p(95)=0.14ms
    vus..................: 1   min=1 max=2
    vus_max..............: 2   min=2 max=2

All k6 tests have finished after 6.27s!

█ Known issues

Note that the following endpoints below have known issues. These tests have either been run with a custom lower threshold limit applied or are quarantined until the issue is fixed:

NAME                | ISSUES
--------------------|-----------------------------------------------------
api_v4_groups_group | https://gitlab.com/gitlab-org/gitlab/-/issues/211504

Full list of issues found both past and present can be found here: https://gitlab.com/gitlab-org/gitlab/-/issues?label_name%5B%5D=Quality%3Aperformance-issues

█ Results summary

* Environment:                10k
* Environment Version:        13.0.0-pre `c1b0cf957e8`
* Option:                     5s_2rps
* Date:                       2020-04-30
* Run Time:                   6.27s (Start: 15:52:45 UTC, End: 15:52:51 UTC)
* GPT Version:                v1.2.6

❯ Overall Results Score: 0.0%

NAME                | RPS | RPS RESULT | TTFB AVG | TTFB P90 | REQ STATUS | RESULT
--------------------|-----|------------|----------|----------|------------|-----------------
api_v4_groups_group | 2/s | -          | -        | -        | -          | FAILED³

³ Failure may not be clear from summary alone. Refer to the individual test's full output for further debugging.

█ Results files

k6/results/10k_v13-0-0-pre_2020-04-30_165245/10k_v13-0-0-pre_2020-04-30_165245_results_output.log
k6/results/10k_v13-0-0-pre_2020-04-30_165245/10k_v13-0-0-pre_2020-04-30_165245_results.json
k6/results/10k_v13-0-0-pre_2020-04-30_165245/10k_v13-0-0-pre_2020-04-30_165245_results.txt

Merge request reports