Automatically install the GitLab for Jira Connect App from GitLab

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

TODO

  • Meet with @lvanc to discuss UX.
  • Determine where success/error state is checked.
  • Determine where an installed connect app can be checked for (either somewhere existing, or else in new persisted state).
  • Weight issue

About

There are two parts to the GitLab/Jira integration. On the Jira side there is the "Jira Connect App" and on GitLab there is the "Jira Integration". We see a disparity between installation numbers where the Jira Connect app has thousands or tens-of-thousands of installs while the GitLab Jira integration has millions of projects using it. We would like to automatically install the Jira Connect App when a user sets up the GitLab integration.

Discovery

A discovery issue investigated the feasibility of this feature: #299482 (closed).

Technical

The install happens asynchronously (even the Jira API triggers the install, which happens later).

The Jira API docs to follow are https://developer.atlassian.com/platform/marketplace/registering-apps/#installing-an-app-using-the-rest-api.

See below for some example Ruby code on initiating the install.

Restrictions

As mentioned in the discovery issue #299482 (comment 522605619) we can only successfully install the app for Jira projects that:

  1. Are on Jira Cloud.
  2. Have development mode enabled in their Manage App settings.
  3. Have not previously installed the app via the MarketPlace.

Breakdown

Keeping state in JiraTrackerData backend

backend-weight1

Add a new enum to track the async progress of the remote install and record any error message.

  • DB migrations to add two columns to jira_tracker_data, one for an enum and one to save any error messages.
  • Add enum and state_machine to JiraTrackerData model (states to signal ready, in_progress, error).
    • transitioning to ready and in_progress should reset any error message recorded.

Jira integration form view frontend

backend-weight2

See #323880 (designs). The Jira integration form section changes what it displays depending on whether the GitLab for Jira app has been installed, or not.

We also display messages in the banner at the top of the page:

  • After create or update:
    • if the install is in progress we display an in-progress message.
    • otherwise, if the app is not already installed (TODO, finding out where to find this data in Slack we prompt to install

At the time of writing the designs also show displaying an error state #323880 however this might not be simple for MVC1 if we want to avoid displaying this message all the time. If it is too hard, for MVC1 the error state would be displayed in the form section only.

We only display the form section and flash messages if:

  • A new feature flag is enabled (one that the backend will also use, so we should co-ordinate).
  • The service's tracker data deployment_type is cloud.

Backend triggers the install backend

backend-weight2

The frontend hits a new route and controller action to trigger the backend to install the app.

The controller action check the new feature flag is enabled (one that the frontend will also use, so we should co-ordinate) to protect against the unlikely scenario where people attempt to hit this endpoint directly, and do nothing if it is not enabled.

It calls a new service, which would:

  • transition the JiraTrackerData stateful property to in_progress.
  • queue a new worker to make the API request.

Then the controller would redirect to the form.

The worker would:

  • make the request to the Jira API to install the app (see example Ruby code below). This will trigger Jira to install the app (asynchronously, so we don't know the result yet)
  • likely need to poll (probably a decaying poll t * 1.5 or something) to check the install state until the install is finished. Although we should get a callback to JiraConnect::EventsController#installed on success, it's possible for the install to fail some times (mentioned in #299482 (comment 522605619) when the Jira does not have development mode enabled, or, if the Jira project has previously had a GitLab for Jira app installed through the Marketplace).
    • if success:
      • transition the stateful property on JiraTrackerData to ready.
    • if error:
      • transition the stateful property on JiraTrackerData to error
      • record the error message on JiraTrackerData
      • log the error?

Example Ruby code

service = JiraService.first # Note: service must be for Jira cloud

# Step 1 of https://developer.atlassian.com/platform/marketplace/registering-apps/#installing-an-app-using-the-rest-api

response = service.client.request(:get, '/rest/plugins/1.0/?os_authType=basic', '', { 'Accept': 'application/vnd.atl.plugins.installed+json' })

response.ok? # => true/false

# Step 2 of https://developer.atlassian.com/platform/marketplace/registering-apps/#installing-an-app-using-the-rest-api
token = response.headers["upm-token"]

plugin_url = JiraConnect::AppDescriptorController::HOME_URL + ::Gitlab::Routing.url_helpers.jira_connect_app_descriptor_path

body = {
  'pluginUri': plugin_url,
  'pluginName': 'GitLab for Jira' # or, Atlassian::JiraConnect.app_name
}.to_json
headers = {
  'Accept': 'application/json',
  'Content-Type': 'application/vnd.atl.plugins.install.uri+json'
}

response = service.client.request(:post, "/rest/plugins/1.0/?token=#{token}", body, headers)

json = JSON.parse(response)

# We would expect `state.done` to be `false` as Jira attempts the install asynchronously

# Example payload:

{"type"=>"INSTALL",
 "pingAfter"=>300,
 "status"=>{"done"=>false, "statusCode"=>200, "contentType"=>"application/vnd.atl.plugins.install.downloading+json", "source"=>"GitLab for Jira", "name"=>"GitLab for Jira"},
 "links"=>{"self"=>"/rest/plugins/1.0/pending/<id>", "alternate"=>"/rest/plugins/1.0/tasks/<id>"},
 "timestamp"=>1614127830034,
 "accountId"=>"<id>",
 "id"=>"<id>"}

# We can poll for the status 

status = JSON.parse(service.client.request(:get, json['links']['self'], ''))

# This is `status` when the GitLab for Jira app has been previously installed through the Marketplace:

{"type"=>"INSTALL",
 "pingAfter"=>300,
 "status"=>
  {"done"=>true,
   "statusCode"=>200,
   "contentType"=>"application/vnd.atl.plugins.task.install.err+json",
   "subCode"=>"connect.install.error.local.override",
   "errorMessage"=>"The app \"gitlab-jira-connect-gitlab.com\" could not be installed as a local app as it has previously been installed from Atlassian Marketplace",
   "source"=>"GitLab for Jira",
   "name"=>"GitLab for Jira"},
 "links"=>{"self"=>"/rest/plugins/1.0/pending/<id>", "alternate"=>"/rest/plugins/1.0/tasks/<id>"},
 "timestamp"=>1614128076006,
 "accountId"=>"<id>",
 "id"=>"<id>"}

# TODO discover the error when the Jira project does not have development mode enabled
Edited by 🤖 GitLab Bot 🤖