WIP: PoC MR: Implement test failure history MVC
What does this MR do?
This is the PoC work for #241759 (closed)
This uses Postgres as the storage option instead of Redis.
To save some time implementing this PoC, I didn't add the specs yet, but tested this manually locally for now. But this is already pretty much the implementation.
Will leave some more notes on the diffs.
Screenshots
Given this sample failing specs on my local MR:
This is the matching JSON response from test_reports.json
:
{
"status": "failed",
"summary": {
"total": 4,
"resolved": 0,
"failed": 2,
"errored": 0
},
"suites": [
{
"name": "jest",
"status": "failed",
"summary": {
"total": 4,
"resolved": 0,
"failed": 2,
"errored": 0
},
"new_failures": [
{
"status": "failed",
"name": "Dog #eat 2 eats",
"classname": "Dog #eat 2 eats",
"execution_time": 0.0,
"system_output": "assert.equal(received, expected)\n\nExpected value to be equal to:\n \"Failed Eat\"\nReceived:\n \"Eat\"\n\nDifference:\n\n- Expected\n+ Received\n\n- Failed Eat\n+ Eat\n at Object.<anonymous> (/builds/root/test-jest/src/dog.test.js:29:14)\n at Object.asyncJestTest (/builds/root/test-jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)\n at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:47:12\n at new Promise (<anonymous>)\n at mapper (/builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:30:19)\n at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:77:41\n at processTicksAndRejections (internal/process/task_queues.js:93:5)",
"stack_trace": null,
"recent_failures_count": 1
}
],
"existing_failures": [
{
"status": "failed",
"name": "Dog #jump 2 jumps",
"classname": "Dog #jump 2 jumps",
"execution_time": 0.004,
"system_output": "assert.equal(received, expected)\n\nExpected value to be equal to:\n \"Jumpssss\"\nReceived:\n \"Jump\"\n\nDifference:\n\n- Expected\n+ Received\n\n- Jumpssss\n+ Jump\n at Object.<anonymous> (/builds/root/test-jest/src/dog.test.js:15:14)\n at Object.asyncJestTest (/builds/root/test-jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)\n at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:47:12\n at new Promise (<anonymous>)\n at mapper (/builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:30:19)\n at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:77:41\n at processTicksAndRejections (internal/process/task_queues.js:93:5)",
"stack_trace": null,
"recent_failures_count": 3
}
],
"resolved_failures": [],
"new_errors": [],
"existing_errors": [],
"resolved_errors": []
}
]
}
Note the added recent_failures_count
under each test case entity. This is the total count of failures for the given spec for the past 14 days across all branches including the MR's. This new attribute will then be used by the frontend to display the new tooltip.
This was the state of the database during this testing (1490001 is "Dog #jump 2 jumps", 1490004 is "Dog #eat 2 eats"):
gitlabhq_development=# select * from ci_test_case_failures ctf join ci_test_cases ctc on ctc.id = ctf.test_case_id;
id | failed_at | test_case_id | build_id | ref_path | id | project_id | key_hash
---------+-------------------------------+--------------+----------+----------------------+---------+------------+------------------------------------------------------------------
1380842 | 2020-10-02 00:51:17.037865+08 | 1490001 | 991 | refs/heads/test-bump | 1490001 | 25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
1380843 | 2020-10-02 01:10:11.005255+08 | 1490001 | 992 | refs/heads/master | 1490001 | 25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
1380844 | 2020-10-02 01:13:51.184806+08 | 1490004 | 993 | refs/heads/test-mr | 1490004 | 25 | 3a9c182cd104600c5ab395b9aa75f791b81b6535de079756213e891983ae6d5e
1380845 | 2020-10-02 01:13:51.184806+08 | 1490001 | 993 | refs/heads/test-mr | 1490001 | 25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
(4 rows)
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team