Skip to content

Automate document with the list of endpoints

Nailia Iskhakova requested to merge automate-tests-docs into master

As we're extending Performance Toolkit and will be covering more endpoint gaps in the future, we need to consider:

  1. How to group and run tests

    • If we add all endpoints from first and second investigations, our test run time will increase significantly. Maybe consider creating a folder/tag like smoke test for all popular endpoints that should be tested regularly and regression for everything else. Some endpoints are being called not very often, but they're slow, so it would be good to keep an eye on them. WDYT?
  2. How to generate test documentation with the list of current tests, so that we don't have to update it manually each time the test is added.

    • I was learning about local CI runners and pipelines, and created this MR as a proposal on how we can do it. It's not polished, but I would be glad to hear your thoughts

    Changes:

  • Group tests in folders by type: api, gitaly, web, quarantined , standalone
  • Each test should have information about it in a multiline comment like so(added to 3 tests in the current example):
    @endpoint: `GET /groups/:id`
    @description: [Get all details of a group](https://docs.gitlab.com/ee/api/groups.html#details-of-a-group)
  • Script collect-test-info parses each test for the fields above and builds a table with results
  • Results are being sent in Wiki by running in pipeline. This script is added to the Merge Request check stage as we only adding tests with MR and so we won't need to update documentation on schedule. On the other hand, not every MR is adding a test, so it may be overkill.
  • Example of generated doc: https://gitlab.com/gitlab-org/quality/performance/wikis/current-test-details

Closes #71 (closed)

Edited by Nailia Iskhakova

Merge request reports