Add attachment attribute to a test case

Part of #202114 (closed)

What does this MR do?

Following up !26788 (merged) where we changed how we persist screenshot files names on test failures.

This MR adds a new attribute to a TestCase object.

• It parses an ATTACHMENT tag if present on our JUnit xml reports
• It creates a new attachment attribute for it

Why are we adding this?

Today's problem

It's not possible to link a screenshot to a test failure. If a failure happens we need to inspect the job artifacts manually which takes time and it's a hard task for us internally.

Solution

We want to make it simple for the user to display a screenshot associated to a test failure. This way we can show a failing screenshot on a pipeline without browsing all the job artifacts manually.

Check the design for this feature.

Screenshots

Here a sample JUnit report this feature will start supporting:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="1" skipped="0" failures="1" errors="0" time="0.017557" timestamp="2020-03-09T15:16:55-04:00" hostname="Maximes-MacBook-Pro.local">
  <properties>
    <property name="seed" value="9386"/>
  </properties>
  <testcase classname="spec.hello_world_spec" name="the home page return hello world" file="./spec/hello_world_spec.rb" time="0.016340">
    <failure message="" type="RuntimeError">Failure/Error: raise
RuntimeError:
./spec/hello_world_spec.rb:8:in `block (2 levels) in &lt;top (required)&gt;'</failure>
    <system-out>[[ATTACHMENT|tmp/capybara/screenshot_return-hello-world.html]]</system-out>
  </testcase>
</testsuite>

This new attachment attribute will then be exposed to our API as part of the JSON reponse of the test_report endpoint.

I'm planning to add it as part of the TestCaseEntity object like this:

expose :attachment, if: -> (test_case) { test_case.has_attachment? }

Then my plan is to add a URL params to our TestSuiteEntity in order to fetch all test cases with attachments:

  expose :test_cases, using: TestCaseEntity do |test_suite, options|
    if options[:attachment]
      test_suite.test_cases.values.flat_map(&:values).select { |test_case| test_case.has_attachment? }
    else
      test_suite.test_cases.values.flat_map(&:values)
    end
  end

Here an example payload of the test_report endpoint this feature will render:

{
  "total_time": 0.01634,
  "total_count": 1,
  "success_count": 0,
  "failed_count": 1,
  "skipped_count": 0,
  "error_count": 0,
  "test_suites": [
    {
      "name": "test",
      "total_time": 0.01634,
      "total_count": 1,
      "success_count": 0,
      "failed_count": 1,
      "skipped_count": 0,
      "error_count": 0,
      "test_cases": [
        {
          "status": "failed",
          "name": "the home page return hello world",
          "classname": "spec.hello_world_spec",
          "execution_time": 0.01634,
          "system_output": "Failure/Error: raise\nRuntimeError:\n./spec/hello_world_spec.rb:8:in `block (2 levels) in <top (required)>'",
          "stack_trace": null,
          "attachment": "tmp/capybara/screenshot_return-hello-world.html"
        }
      ]
    }
  ]
}

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team