Skip to content

Integrate k6 for Integrated Testing

Problem to solve

As the first step for the Integrated Load Testing MVC, we should integrate k6 which is already used by the Quality team for performing load testing against GitLab itself.

Intended users

The intended users would include Personas such as Devon, DevOps Engineer as well as the internal customer of the Quality team.

Further details

This contributes to our vision for advancing the ~"Category:Load Testing" to minimal

Plan

Per https://gitlab.slack.com/archives/C3MAZRM8W/p1583188543124000

  • Quality @gl-quality/enablement-qe will take on moving this feature to maturityminimal we will lean on the Backend Engineer for guidance.
  • Scope the work to backend (minimal ui work)
  • Have a new performance/load type configuration driven by the existing gitlab-ci.yml
  • YAML file takes in k6 configuration as desired and points to test script.
  • We will take the learnings from the original proposal and see where we can fit this into maturityminimal
  • Sid: @james Heimbuck do you think Mek his group can contribute this feature? Cc @mek 👍
  • James Heimbuck: I’m sure we can build on some of the work they have done for the performance testing tool. i think we can have more impact on IACV by using that time to focus on other categories though. *Sid: @james Heimbuck that makes sense. @mek when you get back maybe you can comment on the feasibility of the quality group adding k6 in a minimal form to auto DevOps
  • mek: @james Heimbuck @sid I think this is feasible, as long as its Ruby code, we can pick it up and contribute. We know our usecase and have a good working relationship with the K6 folks. We will likely scope the work to backend (minimal ui work). E.g. a new performance job type configuration driven by the existing gitlab-ci.yml file that can load test given endpoints/urls. If help is needed we can lean on Backend engineers for guidance.
  • Sid @mek that would be awesome!

Previous Proposal

load-testing:
  stage: performance
  image: 8.15.1-slim
  variables:
    URL: https://example.com
    COUNT: 10
    NTIMES: 20
  script:
    - docker pull loadimpact/k6:0.25.1
    - k6 run -u 10 -d 30s script.js
  artifacts:
    reports:
      load: out.json
  • TBD how to use the k6 docker image
  • Do we need to transform the out.json somehow?
  • TBD how k6 results look.

Example k6 output

{ 
   "type":"Metric",
   "data":{ 
      "type":"gauge",
      "contains":"default",
      "tainted":null,
      "thresholds":[ 

      ],
      "submetrics":null
   },
   "metric":"vus"
}{ 
   "type":"Point",
   "data":{ 
      "time":"2017-05-09T14:34:45.625742514+02:00",
      "value":5,
      "tags":null
   },
   "metric":"vus"
}{ 
   "type":"Metric",
   "data":{ 
      "type":"trend",
      "contains":"time",
      "tainted":null,
      "thresholds":[ 
         "avg\u003c1000"
      ],
      "submetrics":null
   },
   "metric":"http_req_duration"
}{ 
   "type":"Point",
   "data":{ 
      "time":"2017-05-09T14:34:45.239531499+02:00",
      "value":459.865729,
      "tags":{ 
         "group":"::my group::json",
         "method":"GET",
         "status":"200",
         "url":"https://httpbin.org/get"
      }
   },
   "metric":"http_req_duration"
}

Permissions and Security

The feature will not introduce any new concepts to the permission and security model - which will be the same as it is for any aspect of the .gitlab-ci.yml

Documentation

Will require a new documentation section for Integrated Load Testing.

Testing

For 👁, was initially overlooked in quad-planning due to failure to account for SET in the dev role, but has been thoroughly reviewed. An integration like this would require:

  • Additional unit/integration level tests ✅
  • Good documentation ✅
  • package-and-qa not necessary but captured E2E testing requirement

Risks

  • Most significant risk is possibility of shared runners unable to handle the load. This is called out specifically in the added documentation. It is expected most users who desire to enable load testing will have the experience to account for this and take note of this call out in the documentation to address it appropriately through runner setup.

What does success look like, and how can we measure that?

This issue will measure usage: https://gitlab.com/gitlab-org/gitlab-ee/issues/10687

What is the type of buyer?

The same buyer as the rest of ~"performance testing" (Director of Release Management, Director of DevOps, etc.) thus this feature is included in GitLab Premium

Links / references

https://docs.k6.io/docs/results-output#section-json-output

Edited by Zeff Morgan