Stop returning alerts from prometheus alerts endpoint
What does this MR do and why?
Implements #372115 (closed)
Do not return any JSON response from the Prometheus alerts endpoint.
When Prometheus sends an alert it does not expect any returning values. We can safely drop the JSON response.
How to verify locally
Prometheus
- Setup [Prometheus Integration]https://docs.gitlab.com/ee/operations/incident_management/integrations.html#http-endpoints) via Settings > Monitor > Alerts
- Clone https://gitlab.com/splattael/gitlab-manual-prometheus
- Edit
gitlab-manual-prometheus/alertmanager.yml
:
global: {}
receivers:
- name: gitlab
webhook_configs:
- send_resolved: true
# e.g. url: http://192.168.10.38:3001/manual-prometheus/autodevops-deploy/prometheus/alerts/notify.json
url: http://localhost:3000/gitlab-org/gitlab-test/prometheus/alerts/notify.json
http_config:
# e.g. bearer_token: 2195dc9c136400428d0124805e78a7bb
bearer_token: c34fc9a8840666ba815429a2af6dab94
route:
group_interval: 5s
group_wait: 5s
receiver: gitlab
repeat_interval: 5m
-
gdk stop prometheus
(if running) to allow our prometheus to be bind on port 9090 -
docker-compose up
and wait ~10 seconds - See http://localhost:3000/gitlab-org/gitlab-test/-/alert_management/2/details
- Via
curl
$ curl -v --request POST --data '{"version":"4","receiver":"gitlab","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"title","gitlab":"hook","gitlab_alert_id":""},"annotations":{},"startsAt":"2018-09-24T08:57:31.095725221Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://prometheus-prometheus-server-URL"}],"groupLabels":{"alertname":"title"},"commonLabels":{"alertname":"title","gitlab":"hook","gitlab_alert_id":null},"commonAnnotations":{},"externalURL":"","groupKey":"{}:{alertname='title'}"}
' --header "Authorization: Bearer TOKEN" --header "Content-Type: application/json" http
://localhost:3000/gitlab-org/gitlab-test/prometheus/alerts/notify.json
# => 201 Created
HTTP Endpoint
This endpoints still returns information for created alerts
- Setup HTTP Endpoint via Settings > Monitor > Alerts
- Fire an alert via
curl
$ curl -v --request POST --data '{ "title": "Testing" }' --header "Authorization: Bearer TOKEN" --header "Content-Type: application/json" http://localhost:3000/gitlab-org/gitlab-test/alerts/notify/test/ID.json
- Request succeeds and returns
[{"iid":1,"title":"Testing"}]
- http://localhost:3000/gitlab-org/gitlab-test/-/alert_management/1/details is available
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Merge request reports
Activity
changed milestone to %15.6
assigned to @ck3g
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Drew Blessing ( @dblessing
) (UTC-5, 7 hours behind@ck3g
)Vasilii Iakliushin ( @vyaklushin
) (UTC+2, same timezone as@ck3g
)To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Dangerremoved [deprecated] Accepting merge requests label
added 1 commit
- 845232a7 - Stop returning alerts from prometheus alerts endpoint
Allure report
allure-report-publisher
generated test report!e2e-review-qa:
test report for 53b9c7c6expand test summary
+------------------------------------------------------------+ | suites summary | +-------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------+--------+--------+---------+-------+-------+--------+ +-------+--------+--------+---------+-------+-------+--------+ | Total | 0 | 0 | 0 | 0 | 0 | ➖ | +-------+--------+--------+---------+-------+-------+--------+
- Resolved by Peter Leitzen
marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed
@sgarg_gitlab could you please review this one? Thanks
requested review from @sgarg_gitlab
- Resolved by Peter Leitzen
- Resolved by Peter Leitzen
- Resolved by Peter Leitzen
Hi, Everything looks great to me
Just some minor comments please check
removed review request for @sgarg_gitlab
requested review from @sgarg_gitlab
@sgarg_gitlab
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.
For more info, please refer to the following links:
added 652 commits
-
845232a7...89a3da03 - 651 commits from branch
master
- 1f9cdf20 - Stop returning alerts from prometheus alerts endpoint
-
845232a7...89a3da03 - 651 commits from branch
requested review from @splattael
removed review request for @sgarg_gitlab
- Resolved by Peter Leitzen
enabled an automatic merge when the pipeline for 8d7fa6de succeeds
added 844 commits
-
1f9cdf20...e5c9b7f4 - 843 commits from branch
master
- 53b9c7c6 - Stop returning alerts from prometheus alerts endpoint
-
1f9cdf20...e5c9b7f4 - 843 commits from branch
enabled an automatic merge when the pipeline for cbad5b7b succeeds
mentioned in commit 8d88677b
added workflowstaging-canary label and removed workflowready for development label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added releasedcandidate label
mentioned in issue #372115 (closed)
mentioned in merge request kubitus-project/kubitus-installer!1646 (merged)
added releasedpublished label and removed releasedcandidate label