Investigate spikes in response time for Unleash endpoints
Problem
While we were investigating #365575 (closed), we noticed that the database queries on Unleash endpoints sometimes become slow. The affected endpoints so far:
GET /api/:version/feature_flags/unleash/:project_id/client/features
POST /api/:version/feature_flags/unleash/:project_id/client/metrics
- https://log.gprd.gitlab.net/goto/8a7bb250-542c-11ed-b0ec-930003e0679c
- https://log.gprd.gitlab.net/goto/8ee6ad90-542c-11ed-8d37-e9a2f393ea2a
In #378863 (closed), we removed some unnecessary queries and still observed latency spikes after the change:
These queries are executed by authorize_by_unleash_instance_id!
. These queries have already been properly indexed, however, the occasional spike happens.
Proposal
- Investigate if this is correlated to
ACCESS EXCLUSIVE
explicit locking. For example,VACUUM FULL
might be related. - Investigate any other ideas that come to mind
Edited by Alishan Ladhani