Skip to content

WIP: PoC MR: Implement test failure history MVC

Erick Bajao requested to merge eb-poc-test-failures-rdb into master

What does this MR do?

This is the PoC work for #241759 (closed)

This uses Postgres as the storage option instead of Redis.

To save some time implementing this PoC, I didn't add the specs yet, but tested this manually locally for now. But this is already pretty much the implementation.

Will leave some more notes on the diffs.

Screenshots

Given this sample failing specs on my local MR: Screen_Shot_2020-10-02_at_1.18.29_AM

This is the matching JSON response from test_reports.json:

{
    "status": "failed",
    "summary": {
        "total": 4,
        "resolved": 0,
        "failed": 2,
        "errored": 0
    },
    "suites": [
        {
            "name": "jest",
            "status": "failed",
            "summary": {
                "total": 4,
                "resolved": 0,
                "failed": 2,
                "errored": 0
            },
            "new_failures": [
                {
                    "status": "failed",
                    "name": "Dog #eat 2 eats",
                    "classname": "Dog #eat 2 eats",
                    "execution_time": 0.0,
                    "system_output": "assert.equal(received, expected)\n\nExpected value to be equal to:\n  \"Failed Eat\"\nReceived:\n  \"Eat\"\n\nDifference:\n\n- Expected\n+ Received\n\n- Failed Eat\n+ Eat\n    at Object.<anonymous> (/builds/root/test-jest/src/dog.test.js:29:14)\n    at Object.asyncJestTest (/builds/root/test-jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)\n    at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:47:12\n    at new Promise (<anonymous>)\n    at mapper (/builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:30:19)\n    at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:77:41\n    at processTicksAndRejections (internal/process/task_queues.js:93:5)",
                    "stack_trace": null,
                    "recent_failures_count": 1
                }
            ],
            "existing_failures": [
                {
                    "status": "failed",
                    "name": "Dog #jump 2 jumps",
                    "classname": "Dog #jump 2 jumps",
                    "execution_time": 0.004,
                    "system_output": "assert.equal(received, expected)\n\nExpected value to be equal to:\n  \"Jumpssss\"\nReceived:\n  \"Jump\"\n\nDifference:\n\n- Expected\n+ Received\n\n- Jumpssss\n+ Jump\n    at Object.<anonymous> (/builds/root/test-jest/src/dog.test.js:15:14)\n    at Object.asyncJestTest (/builds/root/test-jest/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)\n    at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:47:12\n    at new Promise (<anonymous>)\n    at mapper (/builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:30:19)\n    at /builds/root/test-jest/node_modules/jest-jasmine2/build/queueRunner.js:77:41\n    at processTicksAndRejections (internal/process/task_queues.js:93:5)",
                    "stack_trace": null,
                    "recent_failures_count": 3
                }
            ],
            "resolved_failures": [],
            "new_errors": [],
            "existing_errors": [],
            "resolved_errors": []
        }
    ]
}

Note the added recent_failures_count under each test case entity. This is the total count of failures for the given spec for the past 14 days across all branches including the MR's. This new attribute will then be used by the frontend to display the new tooltip.

This was the state of the database during this testing (1490001 is "Dog #jump 2 jumps", 1490004 is "Dog #eat 2 eats"):

gitlabhq_development=# select * from ci_test_case_failures ctf join ci_test_cases ctc on ctc.id = ctf.test_case_id;
   id    |           failed_at           | test_case_id | build_id |       ref_path       |   id    | project_id |                             key_hash
---------+-------------------------------+--------------+----------+----------------------+---------+------------+------------------------------------------------------------------
 1380842 | 2020-10-02 00:51:17.037865+08 |      1490001 |      991 | refs/heads/test-bump | 1490001 |         25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
 1380843 | 2020-10-02 01:10:11.005255+08 |      1490001 |      992 | refs/heads/master    | 1490001 |         25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
 1380844 | 2020-10-02 01:13:51.184806+08 |      1490004 |      993 | refs/heads/test-mr   | 1490004 |         25 | 3a9c182cd104600c5ab395b9aa75f791b81b6535de079756213e891983ae6d5e
 1380845 | 2020-10-02 01:13:51.184806+08 |      1490001 |      993 | refs/heads/test-mr   | 1490001 |         25 | c7104b6fb3748228b03243c498db58efcbf3ebefb532a375c7bb165fccd3d6cf
(4 rows)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Erick Bajao

Merge request reports