GitLab for Jira app: Send installed event for Jira app installed updated for self-managed

This is a part of the epic to make the GitLab for Jira app available for self-managed (&5650 (closed))

The summary of the whole feature is that GitLab.com will act as a proxy for the self-managed instance. This means we will need a JiraConnectInstallation record on .com and self-managed.

The installation record gets created with a webhook that we usually receive from Jira. With the proxy in place, the hook has to come from GitLab.

The hook contains:

• A shared secret that needs to be stored and will be used to authorize future requests.
• A JWT token

Problem with asymmetric JWTs:

A public key is needed to decode the JWT in the installed hook. The key can be fetched from connect-install-keys.atlassian.com/{{KEY_ID}} and KEY_ID is included in the JWT's kid header.

We don't store the JWT token that was included in the installed hook to GitLab.com. So we have to construct a new token, but this won't match a public key at Atlassian.

Solution:

We can offer an endpoint to fetch public keys from on GitLab.com

The security measure in asymmetric JWT is that the domain to fetch the public key is fixed. It points to connect-install-keys.atlassian.com. We can make this an application setting, so users can point it to gitlab.com/-/jira_connect/public_keys instead. This would require:

• An application setting to configure the CDN URL
• A DB table or Redis cache to store public keys
• A route to return public keys

changed milestone to %15.4

added JiraGitLab for Jira Cloud app backend featureaddition groupintegrations [DEPRECATED] typefeature labels

assigned to @Andysoiron

added to epic &5650 (closed)

added sectiondev + 1 deleted label

changed the description

mentioned in issue #372968 (closed)

mentioned in issue #325282 (closed)

mentioned in issue #356361 (closed)

added workflowin dev label

marked this issue as blocking #372968 (closed)

set weight to 2

added missed:15.4 label

changed milestone to %15.5

added missed-deliverable label

added Deliverable label

I was working to make sure slipped issues moved to %15.5 for those we knew would not make the cut in %15.4.

I realize for this one however, it was a new issue we discovered during the milestone. While we didn't already have it planned, it blocked the deliverable. So... I'm not 100% sure how best to treat this one but going with missed-deliverable for now. If we have a better way to treat it, I'm open to ideas.

cc @arturoherrero

@g.hickman Using missed-deliverable could help us a lot when reviewing the backlog to see things we should focus on as they were previously scheduled as a priority. We can keep that label here to raise the attention to this issue.

Good deal @arturoherrero!

mentioned in merge request !98431 (merged)

mentioned in merge request !98437 (merged)

mentioned in epic &5650 (closed)

Setting label(s) Category:Integrations based on ~"group::integrations".

added Category:Integrations label

mentioned in issue gitlab-org/manage/foundations/team-tasks#167

changed milestone to %15.6

added missed:15.5 label

added IntegrationJira label

mentioned in issue #375503 (closed)

marked this issue as blocking #375503 (closed)

mentioned in merge request !101702 (merged)

added workflowin review label and removed workflowin dev label

removed the relation with #375503 (closed)

@Andysoiron What is the status of this? It's been in review for 10 days. What is the remaining weight if this is missed:15.6?

I'm reducing this to 1 point because it was initially 2.

Yes, 1 sounds good @arturoherrero. It is in security review now, but the setup to try the feature locally is so complex that want to come up with a simpler solution before proceeding with the review.

I just created an extra issue for this to make it easier to track the work: #383157 (closed)

@Andysoiron In that case, it looks like #383157 (closed) is blocking this issue and it should be a deliverable.

@arturoherrero I changed it to Stretch and removed it as a blocker as discussed in #383157 (comment 1179350341)

changed milestone to %15.7

added missed:15.6 label

set weight to 1

marked this issue as blocked by #383157 (closed)

mentioned in issue #383157 (closed)

removed the relation with #383157 (closed)

added devopsmanage label and removed 1 deleted label

The Application Security review is now unblocked. I managed to get the setup down to 15 steps. It's not as close to production as the full setup, but it should do the job.

Async update: The Application Security review is still in progress.

@Andysoiron I'm checking !101702 (merged), what do you think about having the maintainer review but with a note to wait for the final security review? I'm trying to speed up things.

Yes, good idea. I just asked @.luke to do the maintainer review. The MR is quite large, and it can be helpful to have a bit of context, so I decided to keep the review within the team.

@Andysoiron Sorry I didn't get to the MR today but I'll review it first thing tomorrow.

mentioned in merge request gitlab-com/www-gitlab-com!110680 (merged)

Async update

We run into a problem with testing the feature. I asked @dcouture to work around this using localhost as GDK host and asked @justin_ho to look into the problem in the meantime.

cc @arturoherrero

@g.hickman The release of the Jira proxy is at risk.

Thanks for raising this @arturoherrero. It looks like the team is working on it, but is there anything else we can be doing to mitigate the risk?

@g.hickman This is looking more positive today. Current status:

• This issue could be closed after !101702 (merged)
• #372968 (closed), merge request ready to be merged from reviewer and maintainer !105888 (merged)
• #382787 (closed), merge request in maintainer review !106263 (merged)

I'll wait for @Andysoiron to double-check things, but we can start more testing very soon.

That's correct @arturoherrero. The MR is merged, and I was able to successfully test this on staging. However, I found that the jira_connect_oauth_self_managed feature needed to be enabled on the self-managed instance, which shouldn't be the case. This is because of the feature check here. It should be easy to remove though.

Next steps are:

1. I'll remove the feature check
2. Compile a few steps to manual QA the full feature and post them in the epic.

Thanks, @Andysoiron and @arturoherrero, great communication on risks!

Based on @Andysoiron's comment #372967 (comment 1202293357), we should complete #359940 (closed) which is also blocked by #355048 (closed).

@Andysoiron, can you update the milestone of the issues, title if necessary (eg. from enabling to removing the feature flag), create new issues if necessary (eg. remove the feature flag after enabling it), and clarify the dependencies to release the feature, please?

I'm reviewing the epic &5650 (closed), and we only have #383157 (closed) and #382787 (closed) for %15.7, but that doesn't reflect the new blockers that you found.

Can you also update the release post thread, please?

@arturoherrero thanks for the ping #383157 (closed) is actually done, I've just closed it.

Regarding the feature flags, we should not default_enable or remove them right now:

#359940 (closed) can be enabled on GitLab.com as soon as %15.7 is released and people are able to update their self-managed instances. Only GitLab %15.7 can be linked to the GitLab.com proxy. We can remove the FF with %15.8.

#355048 (closed) has to stay disabled by default because it breaks the app for self-managed users that didn't set up an OAuth application, but the feature has to be enabled on GitLab.com. I'm not sure how to proceed with this. Maybe we have to wait until %16.0 to remove it and introduce a breaking change. See #355048 (comment 1138032752)

I've updated the release post thread: gitlab-com/www-gitlab-com!110680 (comment 1202969259)

Sorry @Andysoiron, this is still unclear to me as we have "enable" as the title, but we also discuss removing feature flags. Can you update the issues or create new ones with clear milestones about what to do? If we need to enable a feature flag and then remove it, that should be 2 issues to clarify what we have to do and when. Please update the relationship and blockers between them.

@arturoherrero I added a rollout plan to the issue description and changed the label to workflowready for development but the %15.8 milestone is still correct.

This is because the milestone ends on December 17, but the release date is on December 22. We can enable the feature on GitLab.com on the 22nd but technically this is in %15.8 even though we release it with 15.7. Hope this makes sense

@Andysoiron OK, I was confused because you mentioned jira_connect_oauth_self_managed, but that has been fixed !106527 (merged).

mentioned in commit 0a43f05d

closed

I have added steps to test this on staging here: &5650 (comment 1203056855)

added groupimport and integrate label and removed groupintegrations [DEPRECATED] label