Skip to content

Proxy Jira installed events to self-managed

Andy Schoenen requested to merge andysoiron/proxy-jira-connect-installed-events into master

What does this MR do and why?

This is currently blocked by !98437 (merged)

This is part of #372967 (closed). See !96818 (closed) for a full context MR.

When a user installs the GitLab for Jira app we receive an installed hook. It includes a JWT token that we have to verify using a public key. The public key is fetched from connect-install-keys.atlassian.com (see lib/atlassian/jira_connect/jwt/asymmetric.rb:15).

To make the app available for self-managed users, GitLab.com will serve as a proxy. It forwards the installed hook to the self-managed instance, but generates a new JWT token. To make this work, we need to:

  1. Build the JWT infrastructure (!98431 (merged))
    • Add a service to generate JWT tokens.
    • Store the public keys with an expiry date.
    • Provide an endpoint to fetch public keys.
  2. Allow the public key CDN URL to be configured. (!98437 (merged))
    • Add an application setting that defaults to https://connect-install-keys.atlassian.com and can be pointed to https://gitlab.com/-/jira_connect/-/jira_connect/public_keys.
  3. Forward the installed event to self-managed (This MR)
    • Add a service that sends an installed hook to the self-managed instance when instance_url is updated.

I explained the problem in more detail in #372967 (closed)

🔒 Security

When a user installs the GitLab.com for Jira app, Jira sends an installed hook to GitLab.com. This is a handshake to install the app and exchange the secret token. The request is authenticated using an asymmetric JWT.

Currently, the app can only be installed on GitLab.com and we try to make it work for self-managed. The idea is that GitLab.com will serve as a proxy and forward the installation hook to a self-managed instance. Unfortunately, we can't re-use the JWT token that came with the original installation hook because it verifies the URL of the receiver. Therefor, we are building a new JWT that can be verified in the same way as the original JWT from Jira.

The token requires a public key to be decoded. This is usually fetched from https://connect-install-keys.atlassian.com/KEY_ID but self-managed instances need to fetch it from https://gitlab.com/-/jira_connect/public_keys/KEY_ID. For each hook, we generate a new key and store it in REDIS for 5 minutes.

Here is the flow:

  1. User installs the app on GitLab.com
  2. GitLab.com receives an installed with a JWT token
  3. GitLab.com reads the public KEY_ID from the kid header of the JWT token
  4. GitLab.com fetches the public key from https://connect-install-keys.atlassian.com/KEY_ID
  5. GitLab.com decodes the token using the public key
  6. GitLab.com verifies the token
  7. GitLab.com stores the secret_key from the decoded token
  8. User stets the jira_connect_proxy_url application setting to point to https://gitlab.com
  9. User enters their instance_url
  10. GitLab.com stores the instance_url
  11. GitLab.com creates a JWT token with the instance_url as the receiver and stores the public key in REDIS
  12. GitLab.com forwards the secret_key encoded in the JWT token to the instance using an installed hook
  13. Self-managed reads the public KEY_ID from the kid header of the JWT token
  14. Self-managed fetches the public key from https://gitlab.com/-/jira_connect/public_keys/KEY_ID
  15. Self-managed decodes and verifies the token and stores the secret_key
  16. The app is now installed on the self-managed instance

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

To QA the feature in an environment that is close to production, the app needs to be installed on a Jira namespace. This requires to have:

  • 2x publicly available GitLab instances.
  • 1x Jira cloud namespace.

This can be archived using Gitpod. I listed the necessary steps in the Advanced setup section. There is a simpler way to test the feature locally described in the Simple setup section.

Simple setup

To test the feature using your local GDK:

  1. Clone the jira-connect-test-tool git clone git@gitlab.com:gitlab-org/manage/integrations/jira-connect-test-tool.git
  2. Start the app bundle exec rackup. (The app requires your GDK GitLab to be available on http://127.0.0.1:3000.)
  3. On GDK switch to the andysoiron/proxy-jira-connect-installed-events branch.
  4. Open config/gitlab.yml and uncomment the jira_connect config
  5. Restart GDK to pick up new settings gdk restart.
  6. Enable the feature flags: 
    Feature.enable(:jira_connect_oauth)
Feature.enable(:jira_connect_oauth_self_managed)
Feature.enable(:jira_connect_oauth_self_managed_setting)
  7. Go to http://127.0.0.1:3000/-/profile/personal_access_tokens.
  8. Create a new token with the api scope and copy the token
  9. Go to http://localhost:9292
  10. Paste the token and select Install GitLab.com Jira Cloud app.
  11. You should now see the GitLab for Jira Cloud app rendered in an iframe.
  12. Select GitLab (self-managed)
  13. Enter http://127.0.0.1:3000. Usually this should point to a different GitLab instance. For the simple test, we can use the same instance.
  14. Follow the development log tail -f log/development.log
  15. Select Save
  16. The log should now show 3 requests:
    • PUT "/-/jira_connect/installations" This is the request sent by the app frontend to update the installation on the proxy instance.
    • POST "/-/jira_connect/events/installed" This is the request sent by the proxy instance to the self-managed instance to create the installation record
    • GET "/-/jira_connect/public_keys/{{some public key id}}". This is the request sent from the self-managed instance to fetch the public key from the proxy instance.

Advanced setup

You need:

  • 2x GitPod instances running on the proxy-jira-connect-installed-events
  • 1x Jira cloud namespace
  1. Start two publicly available GitLab instances
  2. Go to https://gitlab.com/gitlab-org/gitlab/-/tree/andysoiron/proxy-jira-connect-installed-events
  3. Click GitPod to start a GitPod instance
  4. Do this twice to get two instances running
  5. Wait until the instances are up and running
  6. On each instance go to the gitlab folder cd gitlab
  7. Start a rails console rails c
  8. Enable the feature flags: 
    Feature.enable(:jira_connect_oauth)
Feature.enable(:jira_connect_oauth_self_managed)
Feature.enable(:jira_connect_oauth_self_managed_setting)
  9. Choose one GitPod instance to act as GitLab.com and the other as self-managed
  10. On the GitLab.com instance edit app/controllers/jira_connect/public_keys_controller.rb and remove || !Gitlab.com? from line 13
  11. On the self-managed instance, go to the Ports tab next to Terminal, click the 🌐 button in the port 3000 row to access the instance.
  12. Log in as root or an admin account
  13. Go to https://3000-YOUR_SELF_MANAGED_INSTANCE.gitpod.io/admin/applications
  14. Click New Application
  15. Fill out the fields
    • Name: GitLab for Jira app
    • Redirect URI: https://3000-YOUR_DOTCOM_INSTANCE.ws-eu75.gitpod.io/-/jira_connect/oauth_callbacks (This should point to the other GitPod instance)
    • Scopes: api
    • Make sure Trusted and Confidential are not selected
  16. Click Save Application
  17. Copy Application ID
  18. Go to https://3000-YOUR_SELF_MANAGED_INSTANCE.gitpod.io/admin/application_settings/general
  19. Expand GitLab for Jira App
  20. Paste the Application ID
  21. Set Jira Connect Proxy URL to https://3000-YOUR_DOTCOM_INSTANCE.ws-eu75.gitpod.io
  22. Click Save changes
  23. On both GitPod instances, go to the Ports tab
  24. Click the 🔒 button next to the 3000 port
  25. On Jira cloud got to Apps -> Manage your apps
  26. Enable developer mode
  27. Click Upload app
  28. Paste https://3000-YOUR_DOTCOM_INSTANCE.ws-eu75.gitpod.io/-/jira_connect/app_descriptor.json
  29. Click Upload
  30. Click Get Started
  31. Choose GitLab (self-managed)
  32. Enter the URL of the self-managed instance: https://3000-YOUR_SELF_MANAGED_INSTANCE.gitpod.io
  33. Click Save
  34. On the rails console of the self-managed instance, there should now be one Jira installation JiraConnectInstallation.count

I've recorded a video while writing the steps above: https://youtu.be/qUfQkVrm6uk

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Andy Schoenen

Merge request reports