Update Jira integration to use token-based pagination
What does this MR do and why?
Atlassian has deprecated the /rest/api/2/search
endpoint on Jira Cloud (CHANGE-2046), breaking several GitLab features:
Since Atlassian is rolling out this change gradually, it doesn't affect all users yet, but will eventually impact everyone. Depending on who you ask, this is either severity2 or severity1.
This MR:
- Migrates to the replacement API:
/rest/api/3/search/jql
(docs) - Implements token-based (keyset) pagination instead of offset pagination, replacing page numbers with Previous/Next buttons
- Removes
total_count
since it's no longer available without an additional API call to the approximate count endpoint
Design decisions
The new /rest/api/3/search/jql
endpoint has eventual consistency characteristics and doesn't immediately return newly created issues. Atlassian's recommended approach is to implement a "search and reconcile" pattern that polls the search endpoint and reconciles results with a local cache of expected issues.
Implementing full search-and-reconcile would require significant changes (maintaining a client-side cache of expected issues, implementing reconciliation logic, etc.) and make this MR much larger than it is now.
To address this, a future iteration should implement the full search-and-reconcile pattern. See reconcile-jira-issues.patch for a working implementation which I will add in a follow-up MR (TBA).
References
Screenshots or screen recordings
Before | After |
---|---|
Users will see a error when interacting with the Jira API: ![]() ![]() |
video1206010438 |
How to set up and validate locally
I recommend watching the "after" video above for a rundown.
- In GDK, select a project and configure the Jira integration
- See gdk.test_3000_gitlab-org_gitlab-test_-_settings_integrations_jira_edit for the settings
- If you require Jira Cloud credentials, I'm happy to share mine. Please contact me in Slack.
- View the Jira issues in the project. Use the navigation/search, click into a Jira issue, confirm it all works.
- If you don't have vulnerabilities in the project, use
bundle exec rake gitlab:seed:vulnerabilities
to seed some. - Try to create a Jira issue for a vulnerability. The created issue sometimes appears in the related Jira issues component. If it doesn't you need to perform a hard refresh.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.