WIP: Add finder for prometheus metrics

What does this MR do?

Adds a finder for the standard PrometheusMetrics model. This was a follow-up item from https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/31063#note_200076126.

Updates references to prometheus metrics to use the finder. Note that the PrometheusMetric model used for importing common metrics into the db is different and not related to this work.

Issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/65753

EE MR: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/16235

Does this MR meet the acceptance criteria?

Conformity

• [n/a] Changelog entry for user-facing changes, or community contribution. Check the link for other scenarios.
• [n/a] Documentation created/updated or follow-up review issue created
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Performance and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

changed milestone to %12.3

added Health backend backstage [DEPRECATED] devopsmonitor grouprespond labels

Reviewer roulette

Changes that require review have been detected! A merge request is normally reviewed by both a reviewer and a maintainer in its primary category (e.g. frontend or backend), and by a maintainer in all other categories.

To spread load more evenly across eligible reviewers, Danger has randomly picked a candidate for each review slot. Feel free to override this selection if you think someone else would be better-suited, or the chosen person is unavailable.

Once you've decided who will review this merge request, mention them as you normally would! Danger does not (yet?) automatically notify them for you.

Category	Reviewer	Maintainer
backend	James Edwards-Jones (@jamedjo)	Robert Speicher (@rspeicher)

Generated by Danger

changed the description

changed target branch from enable-specific-embeds to master

added 1182 commits

• cf1f3686...a3b462e9 - 1181 commits from branch master
• b4b9203f - Add finder for prometheus metrics

Compare with previous version

@splattael Could you do a sanity check for me?

My inclination is that the changes in this MR don't improve code clarity/organization/maintainability/etc, and I'm leaning towards scrapping it. Context is in the issue description.

Thanks for taking care of this

It seems we don't have very good documentation on how to write finders

Luckily, @engwan and @nick.thomas did a superb job of guiding me through this when implementing Projects::Prometheus::AlertsFinder

• MR: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9645
• Great discussion on how to simplify code by using boring solutions: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9645#note_144966570

Maybe, this finder can help you to implement Projects::Prometheus::MetricsFinder as well

To compile a quick guide:

• Be specific about arguments passed to the finder (such as project:, group:, metric_id: etc.)
• Organize the finder code along passed arguments (e.g. by_project, by_group, by_metric etc.)
• Make sure the resulting query speedy meaning it has sufficient database indexes (e.g. title and y_label don't have indexes - not sure if they're needed though)
• Add scopes to the PrometheusMetric model to avoid RuboCop violations
• Using .first to find a single record is fine (such as AlertsFinder.new(...).execute.first)
• Finally, find all usages of project.prometheus_metrics and replace them with the new, shiny finder
  • project.prometheus_metrics.where(
  • project.prometheus_metrics.all.group_by(&:group_title).map do |name, metrics|
  • project.prometheus_metrics.each do |project_metric|
  • @metric = project.prometheus_metrics.new # rubocop:disable Gitlab/ModuleWithInstanceVariables
  • metrics = project.prometheus_metrics
  • @metric = project.prometheus_metrics.create(metrics_params) # rubocop:disable Gitlab/ModuleWithInstanceVariables
  • @metric = project.prometheus_metrics.find(params[:id]) # rubocop:disable Gitlab/ModuleWithInstanceVariables
  • @metric = project.prometheus_metrics.find(params[:id]) # rubocop:disable Gitlab/ModuleWithInstanceVariables
  • metric = project.prometheus_metrics.find(params[:id])

I hope this helps

Happy to assist at any time

added 2 commits

• 9892e86b - Update finder to accomodate any metric calls
• 7d7a9c02 - Update app to use new finder

Compare with previous version

added 1 commit

• c054e7d3 - Add support for additional params

Compare with previous version

added 773 commits

• c054e7d3...3441092b - 769 commits from branch master
• 1789fc4e - Add finder for prometheus metrics
• a01757e4 - Update finder to accomodate any metric calls
• fa9e264c - Update app to use new finder
• 75f4446d - Add support for additional params

Compare with previous version

added 1 commit

• ca082c1d - Add finder for prometheus metrics

Compare with previous version

changed the description

EE MR: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/16235

Hey!

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks.

As part of this migration, issues will be moved to the current gitlab-ee project. Merge requests can not be moved, and will be closed and locked instead.

On September 22, 2019 we will start closing all remaining CE merge requests. We have also disabled merging of merge requests in CE.

Here is what you should do to make sure your changes are merged before then:

1. Continue the development process as usual.
2. When the merge request has been reviewed, create a new merge request with these changes at https://gitlab.com/gitlab-org/gitlab-ee/merge_requests.
3. In this new merge request, mention the old merge request.
4. Close the old merge request, and in a comment mention the new merge request.
5. Assign the merge request to the person who last reviewed it, or @ mention them if you are not able to assign the merge request to them.

When creating your EE merge request, you can follow these steps to port over your changes:

1. Add your local copy of GitLab CE as a Git remote to your copy of GitLab EE. This is done using git remote add local-ce path/to/ce/.git.
2. Run git fetch local-ce BRANCH, where BRANCH is the branch containing your CE changes.
3. For every commit in your CE MR, run git cherry-pick COMMIT, with COMMIT being the commit SHA. Squashing your CE commits before may make this process easier.
4. Push these commits to a branch of fork like you normally would when submitting your changes.
5. Submit these changes as a merge request at https://gitlab.com/gitlab-org/gitlab-ee/merge_requests.

More information about this can be found here: https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#cherry-picking-from-ce-to-ee

If you have any questions about all of this, please ask them in our dedicated FAQ issue: https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

• https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
• https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

closed