Integration with Squash TM (a test management solution)
Problem to solve
Some software development teams still rely a lot on manual tests (and, in the future, many will still want to keep some manual acceptance tests). GitLab does not currently fulfil these needs. But some software tools exist for that purpose. Squash TM is such a tool, it is used to manage:
- product requirements
- manual/automated test cases
- the traceability between both
- test campaigns
- defining the tests to be executed (possibly using Gherkin)
- planning the campaign
- reporting the defects in a bug tracker
- tracking the campaign advancement
- reports on the preceding data
It also manages the test automation workflow:
- definition of the tests to be automated
- tracking of the automation progress
- display of the automated test results
Similarly to GitLab, Squash TM is open source (currently https://bitbucket.org/squashtest/squashtest-tm/src/master/, but we plan to move it, as all our proprietary code, in GitLab during Q4 2021 / Q1 2022) and it has a Community and a Premium versions.
(For more information, see https://www.squashtest.com/product-squash-tm.)
An integration between the two products would enable team members to work more effectively than using them side by side:
- The testers and the manager of the test team would be able to access all the data they need in Squash TM to design, plan, and run the tests.
- The testers in charge of automating tests would be able to see in GitLab the necessary information about the tests to be automated.
- The other stakeholders (the product manager, the product owner, the development team lead, the compliance manager…) would be able to see in real-time the test advancement (writing and execution) and results in GitLab.
The first point can be done by using the current GitLab REST API (we already started to work on this):
- The requirements in Squash TM would be references toward the epics and some issues (the ones representing User Stories) of GitLab.
- The iteration and milestone dates defined in GitLab would be accessible in Squash TM in order to help the test team manager to plan the test campaigns.
- The bugs found during the tests would be recorded as issues in GitLab.
The second and third points require that GitLab displays some Squash TM data. This is the purpose of this issue.
User Stories
(These are the user stories only on GitLab side, i.e. the second and third points described above.)
| user story | |
|---|---|
| who | tester in charge of automating a test or a developper in a team using BDD (writing an automated acceptance test before the production code) |
| what | I can see the test definition (possibly in Gherkin) |
| why | so I can write my test |
| user story | |
|---|---|
| who | product manager |
| what | I can see the current progress status of the writing and execution of the test cases |
| why | so I can check that the test team is on track for performing the acceptance tests in time |
| user story | |
|---|---|
| who | development team lead |
| what | I can see the current progress status of the test execution |
| why | so I know than some features are validated and I can expect to not have no rework on them |
| user story | |
|---|---|
| who | compliance manager |
| what | I can see the tests planned and executed, and their results |
| why | so I can check that the tests required for regulatory reasons have been successfully performed |
Intended users
On Squash TM side
- the manager of the test team
- the manual tester
- the automation tester Simone (Software Engineer in Test)
On GitLab side
Proposal
Here is an example of integration we could implement:
- If GitLab Community Edition is integrated with Squash TM Community
- On the issue page (for an Issue which is a User Story), we would display some statistics on the test cases linked to that issue (e.g. the percentage of test cases currently written, the percentage of test cases executed, the percentage of test cases executed successfully; for your information, there is already a similar integration between Jira and Squash TM, https://www.squashtest.com/product-xsquash?lang=en&lightbox=dataItem-knsnibba - or attached image -).
- On the iteration page, we would display the same statistics for the test cases of the issues which are linked to this iteration and for the non-regression tests of the iteration.
- On the milestone page, we would display the same statistics for the test cases of the issues which are linked to this milestone and for the non-regression tests of the milestone.
- If GitLab Premium is integrated with Squash TM Premium
- On the issue page (for an Issue which is a User Story), we would additionally display the details of the test cases linked to that issue (e.g. the list of the test cases, the test steps, the result of the last execution, the result of each step, see https://www.squashtest.com/product-xsquash?lang=en&lightbox=dataItem-knsnibdp - or attached image - for a similar integration between Jira and Squash TM).
- On the iteration page, we would additionally display the same details for the test cases of the issues which are linked to this iteration and for the non-regression tests of the iteration.
- On the milestone page, we would additionally display the same details for the test cases of the issues which are linked to this milestone and for the non-regression tests of the milestone.
This example just aims at describing concretely the type of integration we could have. But, of course, we would discuss with you about the real integration target.
What is the type of the buyer?
- Erin - the Application Development Executive (VP, etc.)
- Dakota - the Application Development Director
Side note
We expect to contribute the code for this integration, under the advice/direction of your architects.