Niko Belokolodov authored 2 years ago and Dylan Griffith committed 2 years ago

Category is not used in the new instrumentation classes
framework, this MR removes it

Niko Belokolodov authored 2 years ago and James Fargher committed 2 years ago

Extend the existing LegacyMetricDecorator class to be able to
store and the submit errors raised during a metric generation

We still have many ignored Rubocop Offenses for
Database/MultipleDatabases in specs. This is a MR
to address validate and fix all of them.

Addressing Issue: #362031

Changelog: other

This fixes all lib spec file offenses

Changelog: added

Mikołaj Wawrzyniak authored 2 years ago and Rémy Coutable committed 2 years ago

ServicePing metrics can take a significant time
and resources to be collected. This commit attempts
to build up tooling that provides insights into
collection time of metrics on self-managed
instances. It adds decorator class to wraps
resulting metrics values. Such approach was
selected because upon collection it is not
known which metric would get value.
Using decorator enables enumeration over
resulting ServicePing payload and
retrieval of collection time information
without to widespread changes to legacy codebase

Changelog: added

Based on the original HLL paper and referenced linear probabilistic
counting paper, this should be m*log(m/V) instead of using the
alpha(m) constant for small cardinality, as linear counting is
used instead.

Luis Mejia authored 3 years ago and Alper Akgun committed 3 years ago

And:

- Update spec for email campaign date
- Update utils specs to check raised error

Remove reference to `ActiveRecord::Base.connection` from the HLL
`BatchDistinctCounter`, and instead use the connection from the
passed-in relation.

  - Add add_metric method in usage_data.rb
  - Update docs
  - Use existing instrumentation classes
  in usage_data.rb

This change fixes some of the MultipleDatabases offenses.

- follow style guide.

The implementation utilizes `WIDTH_BUCKET` SQL function.

Note that this iteration does not use support batching.

Convert histogram keys to strings in Usage Ping

Return a more meaningful histogram fallback

Avoid raising exceptions during usage ping calculations

Return fallback also for invalid statements

Avoid control flow with exceptions

  - From lib/gitlab/utils/usage_data.rb
  - Check is done in Gitlab::UsageDataCounters::HLLRedisCounter
  - Add related feature flag to i_source_code_code_intelligence
  this was missing from the event definition

Deals with  negative numbers and SQL exports

There are instances (e.g. in Gitlab::UsageData) of passing
HLLRedisCounter to the block in #redis_usage_data and it's errors
wouldn't be rescued to return fallback value. This would potentially
result in failing the entire Usage Ping generation.

This prevents that and hardens the method. All HLLRedisCounter custom
exceptions are rescued, since EventError is their superclass.

See !54600

To house all source related operations for Aggregated Metrics
feautre new Sources class has to be added.

It fixes Ruby warnings and removes the files from
deprecations list

Add wrapper method that allow to save selected usage ping metrics
in redis, to use them later on upon aggregated metrics calculation.

Alina Mihaila authored 4 years ago and Mayra Cabrera committed 4 years ago

  - Allow hash options to track_event_in_context
  - Update helpers and tests
  - Use Time.current

Balasankar 'Balu' C authored 4 years ago and Peter Leitzen committed 4 years ago

We introduced `prometheus.server_address` setting in `gitlab.yml` which
is to be Rails application's SSOT regarding a Prometheus address.
Hence, we can deprecate the existing settings `enable` and
`listen_address`.

Changes:
* To `Gitlab::Prometheus::Internal` class methods:
  * `server_address` method renamed to `uri_without_protocol`, to
    reflect what the method actually does.
  * `listen_address` method renamed to `server_address`.
  * `promtheus_enabled?` method now depends on presence of
    `server_address`.

* To `Gitlab::DatabaseImporters::SelfMonitoring::Project::CreateService`
  class methods:
  * `prometheus_listen_address` method renamed to
    `prometheus_server_address`.
  * `internal_prometheus_listen_address_uri` renamed to
    `internal_prometheus_server_address_uri`.

Signed-off-by: Balasankar "Balu" C <balasankarc@autistici.org>

See tracking issue #296169

Mikołaj Wawrzyniak authored 4 years ago and Alper Akgun committed 4 years ago

To enable more flexible operations over hll buckets we need to extract
dedicated class to model their behaviour.

In order to provide designated namespace to house future
clases that will be break apart from counter we need to add dedicated
namespace.

Use new Distributed Postgres HLL counter to calculate challanging
metrics for Usage Ping

  - Now we have the check under base module
    HLLRedisCounter
  - Adjust the tests

Remove usage ping check tests

This does:

- add all leftover `development` feature flags
- ensures that licensed `feature flags` are actually checked
  against `type: :licensed`
- makes `development` to be `optional: false`, aka `required`

  * Allow to track any value
  * Add documentation
  * Add tests

- auto detect Prometheus uri scheme
- skip Prometheus SSL certificate verification