Skip to content

Fix GraphQL pipeline findings pagination

Malcolm Locke requested to merge 441306-security-report-finding-pagination into master

What does this MR do and why?

Fix GraphQL pagination on the pipeline.securityReportFindings resolver.

This requires some fairly in depth reworking explained below:

Rework of SecurityFindingsFinder#limit calculation

This finder was implemented for a REST endpoint where pagination is implemented with page and per_page parameters. It uses offset pagination currently (conversion to keyset is planned). Because of this the pagination boundaries (OFFSET) always fall on equal divisions of per_page size.

                     REST pagination

      page:1 per_page:10           page:2 per_page:10
|----------------------------|-----------------------------|----
 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 ..

The GraphQL cursor based pagination can use arbitrary offsets using first or last arguments to specify page size and before or after to determine the offset

                     GQL pagination

                         first:12 after:5
               |----------------------------------|
 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 ..

The Security::FindingsFinder has undergone extensive customization over time to make it perform acceptably with large data sets. To facilitate this the finder needs to know some information about the current pagination position. Previously it was using the page and per_page parameters from the REST endpoint to determine this, but with this change it has needed to be modified.

Getting pagination information from the GQL resolver

The default GraphQL connection implementation removes the pagination arguments before calling resolve, so the pagination params are not available by default. Using connection_extension: ::Gitlab::Graphql::Extensions::ExternallyPaginatedArrayExtension works around this.

Once these arguments are available they require some manipulation to read. The first and last arguments are plain integers, but before and after are base64 encoded strings. Usually they contain encoded JSON with the keyset pagination parameters, but when using offset_paginate before and after contain the base64 encoded first and last id of the returned record set.

Steps to reproduce

Ensure you have EE features and pipeline runners available on your GDK.

  • git switch master
  • Clone https://gitlab.com/gl-demo-ultimate-myacksmith/482983-test-group/graphql-vulnerability-findings-test into your GDK
  • In rails console set the feature flag to use GraphQL instead of REST on the pipeline security tab Feature.enable(:pipeline_security_dashboard_graphql)
  • Once the project's pipeline has completed, visit 'Pipeline -> Security'
  • The 'Scan details' section should show a count of 164 vulnerabilities'
  • Paginate through the vulnerability list. The full list of findings is not shown.
  • git switch 441306-security-report-finding-pagination
  • You should now be able to paginate through all of the findings.

Related to #441306 (closed)

Edited by Malcolm Locke

Merge request reports