Skip to content

Add quarantine tag message

Dan Davison requested to merge qa-quarantine-tag-reporting into master

What does this MR do?

Instead of our current process of adding a quarantine issue in a comment above the examples, let's utilize RSpec's metadata.

Adds the ability to specify a quarantine message, such as a GitLab Issue.

Before

spec.rb

# https://gitlab.com/gitlab-org/gitlab/issues/12345
it 'can create and associate a project cluster', :quarantine do
  ...
end
Configure
  Kubernetes Cluster Integration
    Project Clusters
      can create and associate a project cluster (PENDING: In quarantine)

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Configure Kubernetes Cluster Integration Project Clusters can create and associate a project cluster
     # In quarantine
     # ./qa/specs/features/browser_ui/7_configure/kubernetes/kubernetes_integration_spec.rb:24

After

spec.rb

it 'can create and associate a project cluster', quarantine: 'https://gitlab.com/gitlab-org/gitlab/issues/12345' do
  ...
end
Configure
  Kubernetes Cluster Integration
    Project Clusters
      can create and associate a project cluster (PENDING: In quarantine: https://gitlab.com/gitlab-org/gitlab/issues/12345)

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Configure Kubernetes Cluster Integration Project Clusters can create and associate a project cluster
     # In quarantine: https://gitlab.com/gitlab-org/gitlab/issues/12345
     # ./qa/specs/features/browser_ui/7_configure/kubernetes/kubernetes_integration_spec.rb:24

Notes:

  • This still allow us to specify :quarantine by itself, but perhaps a future iteration would be to not allow that. I.e, if a test is quarantined, an issue "MUST" be associated.
  • After merging, this will require us to update the quarantine-test-report sent to Slack.

Does this MR meet the acceptance criteria?

Conformity

Edited by Dan Davison

Merge request reports