Grafana/influxdb counting extraneous internal sql queries

Currently we subscribe to ActiveRecord sql events, in order to count number of sql queries per request.

However in the most simple actions, we can observe that it fluctuate quite a bit.

After some investigation, one cause is found, which is that some internal queries are also counted.

Methodology

In the hook Gitlab::Metrics::Subscribers::ActiveRecord#sql, I added a logging line:

          Rails.logger.debug("hihi   #{event.payload[:sql]}\n")

Then in the Rails console, I profiled Project::RawController#show action:

# the file has to be viewable and user needs access token.
results = Gitlab::Profiler.profile('/twitter/typeahead-js/raw/master/CONTRIBUTING.md', user: User.first)

Results

For log see https://gitlab.com/snippets/1750882

It is discovered that for a fresh console, some sort of internal postgresql lookup is triggered, e.g:

               SELECT attr.attname
            FROM pg_attribute attr
            INNER JOIN pg_constraint cons ON attr.attrelid = cons.conrelid AND attr.attnum = any(cons.conkey)
            WHERE cons.contype = 'p'
              AND cons.conrelid = '"projects"'::regclass

For later runs these internal queries are not longer present, probably cached by Rails.

For the initial fresh run, 12 internal queries were present, which is quite a big noise.

Another source of extraneous counts are from begin and commit queries. Those each adds one count.

Proposal

Since these internal queries are noise for developers, we can filter out 'SCHEMA', 'BEGIN' and 'COMMIT'.