Pros & cons of pagination types
Problem to solve
Currently, the Vulnerability Report uses "infinite" scroll to load more results. There are some pros and cons to this infinite scroll functionality:
-
Pros
- A user could keep scrolling down to load more items and then use CTRL+F to search for keywords. This wouldn't be possible with pagination where the shown items are replaced when the page is changed. The real solution here is a search component with free text search which will be explored in Spike: Investigate options for searching vulnerabilities.
-
Cons
- Not currently able to jump back to your spot after a lot of scrolling. If I scroll down the Vulnerability Report and click into a vulnerability, and then hit "back", I'm taken back up to the top of the list. This would be solved with pagination because we'll save the current page. Although we may be able to solve this with infinite scrolling?
- Loading a lot of vulnerabilities and trying to do a bulk update (say 150 are selected, for example) often leads to the page failing/timing out because we have to dismiss each one individually instead of having a true single-request bulk dismiss. Pagination would solve this because you'll only be able to dismiss 20 at most at a time. (Related FYI: we plan on introducing a [bulk select across pages](#350226 (closed) which will require additional backend work.)
- Difficult to jump to the right record if your current filter has 100s or 1000s of items. E.g., if you have 2500 records, you can start from the beginning or the end (by reversing a column's sort order) but to get to anything in the middle, you have to scroll through ~1250 records. This likely can only be improved with page-based pagination where users can fiddle with the query string to jump to specific pages. *see more on this below
In !79834 (merged), @dftian explored moving from infinite scrolling to pagination, but there are many limitations with this solution. Without backend support at the moment, we are unable to explore offset pagination, which seems like would be the ideal UX here. I can't in good UX conscience get behind this cursor-based pagination; it would have potential to cause a poor UX and potentially hazardous situation to have an empty state page when, in fact, there are "hidden" security vulnerabilities on the previous or next page. This may lead to a security engineer thinking they've cleared out vulnerabilities matching the current filter set when in fact they have to navigate to another page. This cursor-based pagination would solve some of our concerns but would also create too many other huge new ones.
JTBDs
- When I'm on the Vulnerability Report and I click into a vulnerability to review it, and then go back to the Vulnerability List, I want to be taken to where I left off in the list so that I can continue reviewing vulnerabilities in the order that I was reviewing them previously.
Problem to solve: we currently scroll the user back up to the top of the page after going back to the list, causing them to have to scroll down and find their place again.
- **When I'm on the Vulnerability Report and I clear out all vulnerabilities on a page (for example, setting all on page 2 to "Dismiss" when the filter is only set to show "Confirmed" and "Needs Triage"), I want the Vulnerability Report to take me to the remaining vulnerabilities in the list that match the current filter set so that I make sure I've reviewed all security risks (**
*
and don't accidentally see an empty state and miss ones on hidden pages).
*
Problem to solve: This is what's happening with cursor-based pagination
- When I dismiss a vulnerability (or vulnerabilities) from the list using the bulk feature, I want those vulnerabilities to be removed from the list when they no longer match the existing filters so that only vulnerabilities that I need to review or triage are left in the list.
Problem to solve: Dismissed vulns stay in the list; being addressed in #337599 (closed)
Next steps
We'll be rolling out this cursor-based/ offset-based pagination behind a feature flag in order to get some feedback from the GitLab AppSec team. If we get specific feedback that the functionality is confusing then we can validate that we shouldn't move forward with this option.
We also need to gather more research on why users are requesting a direct link to a page. My understanding of the use case of this is 1. someone may want to pick up where they left off in a list of triaging vulnerabilities (or whatever it is the list is of) 2. they may want to send the link to a coworker to have a look at the items on that page. @dmoraBerlin just had a customer complain that we removed the page-based pagination which you can watch here: https://dovetailapp.com/tags/1tSwFpBvdCnQwxyunGVbXs if you have Dovetail access. However, he doesn't mention why he wants to jump directly to a different page. It could be a limitation of our poor filter/ search capabilities (related: #342079 (closed)).
As Dominic Couture says in this comment, the functionality to bulk select all across pages matching filters may address the problem more than needing to jump to a given page. My other theory is that users are requesting to jump to a specific page as a workaround to our poor filter and search functionality. When we have better filtering (and search) on the Vulnerability Report, these queries will be saved in the URL (like any page with applied filters or search keyword) which will let users share a more accurate list of vulns with a coworker or to bookmark (or save as a new tab) and return to.
Additional context & cross-stage collaboration
We recently learned that the Issues page on the GitLab project which uses cursor-based pagination (prev / next) buttons) is behind a feature flag and without this FF turned on, it's page-based pagination (1, 2, 3...). @donaldcook explained that this happened mostly because they moved to GraphQL on the issue list, where cursor-based pagination is more performant. @dftian then responded that we were wondering if we're moving to cursor-based pagination site-wide as the thing we're doing going forward, or if we can use offset-based pagination with GraphQL. As Donald mentioned, we don't have offset-based pagination right now for GraphQL endpoints, but we could add in that ability. @digitalmoksha responded that in general we’re moving to cursor-based pagination, primarily for performance/scaling. GraphQL does also support offset pagination, and it’s possible, when implementing the resolver, to choose offset pagination. However, cursor-based is our default. We have a couple of discussions about this in our development docs which should answer most of your questions:
- GraphQL API style guide - Pagination implementation
- GraphQL pagination
- Pagination guidelines
- Pagination performance guidelines
We recently had a customer complain about the removal of the page-based pagination when moved to a cursor-based pagination during a user research session with @dmoraBerlin. View that video in Dovetail here.
Responses in Slack(internal):
Brett Walker: An interesting question is what does it mean to be on the “third” page of a list of vulnerabilities. The “third page” is never static - as you dismiss vulnerabilities (and maybe there are multiple people doing this at the same time), what’s on that third page changes. You can end up skipping over something. With cursor based, it points to a specific point in the list of records. So when you ask to see that cursor again, you will get all the vulnerabilities from that cursor on. That tends to be more accurate. And you can share that link with someone. If they visit that link, with cursor based you’re going to see any items after that cursor, which is probably what the user intended. If you share a paginated link, that just shows whatever happens to be in the third batch of data - much less useful for sharing, particularly if items can be deleted/dismissed. I don’t know why we don’t have this in the standard cursor based pagination controls, but there should be a First and Last button. I don’t think there is any technical reason not to, and it should gives the user the ability to jump to the beginning or end of the list.
Neil: The workflow of vulnerability management has the user click into a vulnerability, act upon it, then click back to return to the result list. It would be unfavorable for them to have to start over and find their position again. As Becka called out, our current infinite scroll, lazy loading solution doesn't achieve the ideal experience either. You can play with it at https://gitlab.com/gitlab-examples/security/security-reports/-/security/vulnerability_report
Brett: If you go back to the same url as the page the vuln was on, then there shouldn’t be a problem - it should be the same list, minus any changes. You can’t guarantee that with page-based. With regard to "It’s so confusing to have an empty state page when there are vulns on the previous or next page", I agree. But I think these edge cases can be handled.
/cc @uhlexsis, Foundations team: @jeldergl @jareko @aregnery, Secure stakeholders: @gitlab-com/gitlab-ux/secure-protect-ux @matt_wilson @nmccorrison @thiagocsf