Set up QA of Slack app on staging

About

We would like to begin QAing changes to our Slack App on staging (using the staging copy created in #363815 (closed)) before we release it.

Slack apps cannot be staged before they are published to the Slack directory. Changes to the app can only be interacted with after being published to the Slack directory for everyone. This has made it hard to QA a change properly before the Slack App is released to the public.

We recently #363815 (closed) created a staging copy of our Slack App that works with our staging environment. We can use the staging copy to test changes we intend to make in the Slack App before releasing it. Note, that unfortunately, the process of maintaining the copy is manual and prone to human error. More documentation about the staging copy is in the process of being added to our internal Slack dev guide.

Proposal

We would like to be able to QA on staging:

  • Slash commands
  • App home feature

The tests should run against:

  • Old (existing) Slack App integration data
  • A fresh create scenario.

Scenarios

Feature: Slack slash commands
  I can use the Slack application execute commands against my GitLab project

Background:
  Given I am authenticated in Slack and GitLab
  And I am the owner of the slack workspace
  And a GitLab Slack application exists in the Slack directory for the environment that my project is in
  And I trigger a Slack app install from the integrations settings page for the slack workspace and clicking "Allow"
  And I click the GitLab app home in Slack
  And I click "Connect your GitLab account" in Slack
  And I click "Authorize" in GitLab
  And I switch tabs to trigger a UI update in Slack
  And the GitLab app home says my account is connected.

Scenario: New issue
  Given I am in a Slack channel
  When I type `/gitlab <project> issue new <title>`
  And I type `shift+return`
  And I type <description>
  And I type `return`
  Then a new issue is made against <project> with the <title> and <description>

Scenario: Show issue
  Given I create a new issue in GitLab [project] with id [id]
  And I am in a Slack channel
  When I enter `/gitlab [project] issue show [id]`
  Then the title and description of the issue will appear in the Slack channel

Scenario: Close issue
  Given I create a new issue in GitLab project [project] with id [id]
  And I am in a Slack channel
  When I enter `/gitlab [project] issue close [id]`
  Then the issue will be closed in the GitLab project

Scenario: Move issue
  Given I create a new issue in GitLab project [project] with id [id]
  And I create a new GitLab project [project2] 
  And I am in a Slack channel
  When I enter `/gitlab [project] issue move [id] to [project2]`
  Then the issue is no longer in [project]
  And the issue is present in [project2]

Scenario: Comment on issue
  Given I create a new issue in GitLab project [project] with id [id]
  And I type `/gitlab [project] issue comment [id]`
  And I type `shift+return`
  And I type `[comment text]`
  And I type `return`
  Then a comment with [comment text] is added to the issue in GitLab

# big questions around this.
Scenario: Deploy

Scenario: Run job
  Given I create a GitLab project [project]
  And the project has a `.gitlab-ci.yml` file with a job named [job] that echos $CHAT_INPUT
  And I am in a Slack channel
  When I enter `/gitlab [project] run [job] [arguments]
  And I view the latest pipeline in [project]
  Then the pipeline job should be [job]
  Then the job output should contain [arguments]
Edited by Sean Gregory