Make rspec_profiling data useful
Problem to solve
We have many specs written for gitlab
project (and still growing) and there is a potential to optimize some of these tests and decrease CI runtime. The problem is that currently there is not an easy way to analyse rspec runtime and find least performant tests which could be (perhaps) optimized.
When running CI for gitlab
project we already gather profiling data about each executed spec (e.g. duration, query count, request count...) and we store this data to DB and make it accessible through redash.gitlab.com. The problem is that currently this database is little bit outdated and probably too big (current ID in spec_profiling_results
table is 126540643) which causes that even simple queries time out on it.
Intended users
Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/
Proposal
It would be nice to clean up and restore this source of profiling data and make it more accessible to developers and community, the major benefit would be that anyone could then easily get a list of potential rspec which may need optimization.
A list of steps to achieve this:
-
create a new Postgres DB (or table in an existing DB) for storing rspec profiling data, ideally in Cloud SQL, -
update RSPEC_PROFILING_POSTGRES_URL CI ENV variable to point to the new database -
update scripts/insert-rspec-profiling-data
(if necessary) to automatically create table in the new DB -
add this new postgres DB as a data source in Sisense/Periscope -
prepare a query which lists top X specs sorted by duration and embed this on our profiling page in handbook -
add additional queries (for analyzing e.g. number of SQL queries) if needed
This might have more potential in future, a couple of ideas:
- include rspec statistics in our performance report displayed in the MR widget
- create a bot which runs periodically and creates an issue if there is a significant rspec duration regression
Permissions and Security
It will be necessary to provide access to the rspec profiling db from our Sisence/Periscope
Documentation
-
make sure our handbook is updated -
check if we should make these data more visible from other places, e.g. from developer documentation