Skip to content

GitLab Next

Closed
Created by Dylan Griffith@DylanGriffithMaintainer

Fix `Ci::PipelinesFinder#by_name` cross-join `users` and `ci_pipelines`

Release notes

In GitLab 14.3, we will remove the ability to filter by name in the list project pipelines API endpoint to improve performance. If you currently use this parameter with this endpoint, you will need to switch to username.

Problem to solve

There is an open MR where we started this discussion already !66020 (closed) but it has stalled.

This Ci::PipelinesFinder#by_name method is only used in the API (see name field in the docs). Please note this is referring to the user's display name (the one you can edit and add whatever you want to and not the username which is unique and harder to change).

This cannot work when ci_* tables are moved to a separate database.

Options

  1. Remove this feature from the API
  2. Change the code to load the candidate users using User.where(name: 'blah').pluck(:id) in the first query and pass this through in the 2nd query. Note the performance considerations discussed in !66020 (comment 625501999) . This approach will have to have some reasonable limit if there is a name that is incredibly common in a GitLab instance we don't want to OOM ourselves by loading all the users with that name into memory.
  3. If we really need to correctly handle this feature then de-normalizing the users.name onto the ci_pipelines is the only robust way to do it. But it's not recommended because this is very expensive as it's a text field and updates very regularly. We do not recommend this de-normalization.

Sharding team recommended solution

Remove the feature from the API. It seems unlikely that people use this feature and it seems like a pointless feature anyway since we already support filtering by username. name is a field that users can change constantly and we don't even have use cases ourselves for filtering by name. More of my reasoning at !66020 (comment 624876082)

Edited by Dov Hershkovitch