Slow API /projects endpoint

Zendesk: https://gitlab.zendesk.com/agent/tickets/98224 Follow-up Zendesk: https://gitlab.zendesk.com/agent/tickets/143790

For this customer, calling the /projects endpoint with various filters is problematic. It may be a specific case with tons of projects because I notice they're calling page 501.

Query example: https://gitlab.example.org/api/v4/projects?archived=false&membership=false&order_by=id&owned=false&page=501&per_page=1&private_token=supersecret&simple=false&sort=asc&starred=false&statistics=false&visibility=internal&with_issues_enabled=false&with_merge_requests_enabled=false

See the ticket above for SQL and HTML profiler output. Below is a screenshot of the relevant portion of the HTML profile. It shows that the bulk of time is spent in API::Entites::Project.forks_counting_projects. Apparently mapping Project#forked_from_project is taking forever. I notice that a whopping 2+ million calls are happening to various ActiveRecord::Attribute*. That's probably a problem.

Screen_Shot_2018-06-20_at_3.20.37_PM

Edited Jan 13, 2020 by Drew Blessing
Assignee Loading
Time tracking Loading