Enable endpoints corresponding to AlertManagement::HttpIntegrations

What does this MR do?

From a user perspective, each AlertManagement::HttpIntegration represents a combination of endpoint & auth token which can accept alert notifications. By default, notifications will be treated as "generic" alerts.

Changes in this MR:

• Adds feature flag for multiple http endpoints
• Creates route which corresponds to AlertManagement::HttpIntegrations.
• Limits multiple HTTP Integrations to GitLab Premium
• Adds finder for HTTP Integrations
• Configures NotificationsController to find active integration corresponding to request
• Allows NotifyService to use a found integration to validate token & create aptly-named system notes

Relationship to AlertsService:

• Existing AlertsService should work as expected, with the feature flag enabled or disabled
• AlertsServices will eventually be migrated to AlertManagement::HttpIntegrations with an endpoint_identifier of 'legacy'. These integrations should continue to accept notifications to their usual endpoint.

Testing

Testing backend functionality

My lazy approach: modify url & token passed to the FE & let FE construct the query.

Pre-req: enable feature flag.

I use project 21 in my testing, so that's the id you'll see below.

Create our integration:

AlertManagement::HttpIntegration.create!(
  project_id: 21,
  name: 'Tester HTTP Integration', 
  active: true
)

Update OperationsHelper with pointer to new integration:

diff --git a/app/helpers/operations_helper.rb b/app/helpers/operations_helper.rb
index 521f394a920..da2aea8f4c5 100644
--- a/app/helpers/operations_helper.rb
+++ b/app/helpers/operations_helper.rb
@@ -24,9 +24,11 @@ module OperationsHelper
       'prometheus_reset_key_path' => reset_alerting_token_project_settings_operations_path(@project),
       'prometheus_authorization_key' => @project.alerting_setting&.token,
       'prometheus_api_url' => prometheus_service.api_url,
-      'authorization_key' => alerts_service.token,
+      # 'authorization_key' => alerts_service.token,
+      'authorization_key' => AlertManagement::HttpIntegration.last.token,
       'prometheus_url' => notify_project_prometheus_alerts_url(@project, format: :json),
-      'url' => alerts_service.url,
+      # 'url' => alerts_service.url,
+      'url' => AlertManagement::HttpIntegration.last.url,
       'alerts_setup_url' => help_page_path('user/project/integrations/generic_alerts.md', anchor: 'setting-up-generic-alerts'),
       'alerts_usage_url' => project_alert_management_index_path(@project),
       'disabled' => disabled.to_s

• Navigate to project > Settings > Operations > Alerts.
• Enable HTTP Endpoint.
• Send test payload.

{
  "title": "This alert should be the first from HTTP Integration"
}

• After success response, navigate to Operations > Alerts.

• Find new alert & see system note indicating alert came from "Tester HTTP Integration"

• Inactive integrations: Update the integration to inactive & see the payload test fail.

• Legacy integrations: Clear changes to OperationsHelper. Create integration from an alerts service. See old url format work.

alerts_service = Project.find(21).alerts_service

AlertManagement::HttpIntegration.create!(
  project_id: 21,
  name: 'WOWOWOW ITS LEGACY', 
  endpoint_identifier: 'legacy',
  token: alerts_service.token,
  encrypted_token: alerts_service.data.encrypted_token,
  encrypted_token_iv: alerts_service.data.encrypted_token_iv,
  active: true
)

Sending payload	Alert detail

Testing database query performance

Setup: alert_management_http_integrations is empty on prod. We'll seed some data & cleanup after.

Seed:

40_000.times do |i|
  AlertManagement::HttpIntegration.create!(
    project_id: [4, 5, 6, 7, 8].sample, 
    name: "Removable Integration #{i}", 
    active: rand(2).positive?
  )
end

Cleanup:

[4, 5, 6, 7, 8].each do |id|
  Project.find(id).alert_management_http_integrations.destroy_all
end

---

Pick an integration to find! Sample trigger for primary queries:

require 'activerecord-explain-analyze'
project = Project.find(4)
endpoint_identifier = '72e6ed9e7bf06d1b'

AlertManagement::HttpIntegrationsFinder.new(
  project,
  endpoint_identifier: endpoint_identifier,
  active: true
).execute.limit(1).explain(analyze: true)

GitLab Core: Only look at the first integration for the project -- 2 queries

Pre-req: disable license/comment out finder override

Sample trigger for nested query:

project.alert_management_http_integrations.limit(1).explain(analyze: true)

SELECT "alert_management_http_integrations".* 
FROM "alert_management_http_integrations" 
WHERE "alert_management_http_integrations"."project_id" = $1 
ORDER BY "alert_management_http_integrations"."id" ASC 
LIMIT $2  
[["project_id", 4], ["LIMIT", 1]]

Depending on which project I queried against, I got two different plans:

Explain analyze visualization: https://explain.depesz.com/s/Dvtv

EXPLAIN for: SELECT "alert_management_http_integrations".* FROM "alert_management_http_integrations" WHERE "alert_management_http_integrations"."project_id" = $1 LIMIT $2
Limit  (cost=0.00..0.18 rows=1 width=161) (actual time=0.015..0.015 rows=1 loops=1)
  Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
  Buffers: shared hit=1
  ->  Seq Scan on public.alert_management_http_integrations  (cost=0.00..1476.11 rows=8007 width=161) (actual time=0.014..0.014 rows=1 loops=1)
        Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
        Filter: (alert_management_http_integrations.project_id = '4'::bigint)
        Rows Removed by Filter: 9
        Buffers: shared hit=1
Planning Time: 0.079 ms
Execution Time: 0.031 ms

Explain analyze visualization: https://explain.depesz.com/s/Xjab

 EXPLAIN for: SELECT "alert_management_http_integrations".* FROM "alert_management_http_integrations" WHERE "alert_management_http_integrations"."project_id" = $1 LIMIT $2
Limit  (cost=0.29..1.40 rows=1 width=161) (actual time=0.005..0.006 rows=1 loops=1)
  Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
  Buffers: shared hit=3
  ->  Index Scan using index_alert_management_http_integrations_on_project_id on public.alert_management_http_integrations  (cost=0.29..9.14 rows=8 width=161) (actual time=0.005..0.005 rows=1 loops=1)
        Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
        Index Cond: (alert_management_http_integrations.project_id = '21'::bigint)
        Buffers: shared hit=3
Planning Time: 0.037 ms
Execution Time: 0.016 ms

Full finder-query output:

Explain analyze visualization: https://explain.depesz.com/s/bA9D

SELECT "alert_management_http_integrations".* 
FROM "alert_management_http_integrations" 
WHERE "alert_management_http_integrations"."project_id" = $1 
AND "alert_management_http_integrations"."id" = $2 
AND "alert_management_http_integrations"."endpoint_identifier" = $3 
AND "alert_management_http_integrations"."active" = $4 
ORDER BY "alert_management_http_integrations"."id" ASC 
LIMIT $5  
[["project_id", 4], ["id", 80032], ["endpoint_identifier", "72e6ed9e7bf06d1b"], ["active", true], ["LIMIT", 1]]

Explain analyze raw output:

 EXPLAIN for: SELECT "alert_management_http_integrations".* FROM "alert_management_http_integrations" WHERE "alert_management_http_integrations"."project_id" = $1 AND "alert_management_http_integrations"."id" = $2 AND "alert_management_http_integrations"."endpoint_identifier" = $3 AND "alert_management_http_integrations"."active" = $4 LIMIT $5
Limit  (cost=0.29..2.31 rows=1 width=161) (actual time=0.007..0.008 rows=0 loops=1)
  Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
  Buffers: shared hit=3
  ->  Index Scan using alert_management_http_integrations_pkey on public.alert_management_http_integrations  (cost=0.29..2.31 rows=1 width=161) (actual time=0.007..0.007 rows=0 loops=1)
        Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
        Index Cond: (alert_management_http_integrations.id = '80032'::bigint)
        Filter: (alert_management_http_integrations.active AND (alert_management_http_integrations.project_id = '4'::bigint) AND (alert_management_http_integrations.endpoint_identifier = '72e6ed9e7bf06d1b'::text))
        Rows Removed by Filter: 1
        Buffers: shared hit=3
Planning Time: 0.059 ms
Execution Time: 0.019 ms

---

GitLab Premium: All integrations are available -- 1 query

Explain analyze visualization: https://explain.depesz.com/s/7T7E

SELECT "alert_management_http_integrations".* 
FROM "alert_management_http_integrations"
WHERE "alert_management_http_integrations"."project_id" = $1 
AND "alert_management_http_integrations"."endpoint_identifier" = $2 
AND "alert_management_http_integrations"."active" = $3 
LIMIT $4  
[["project_id", 4], ["endpoint_identifier", "72e6ed9e7bf06d1b"], ["active", true], ["LIMIT", 1]]

Explain analyze raw output:

 EXPLAIN for: SELECT "alert_management_http_integrations".* FROM "alert_management_http_integrations" WHERE "alert_management_http_integrations"."project_id" = $1 AND "alert_management_http_integrations"."endpoint_identifier" = $2 AND "alert_management_http_integrations"."active" = $3 LIMIT $4
Limit  (cost=0.41..772.81 rows=1 width=161) (actual time=0.090..0.090 rows=1 loops=1)
  Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
  Buffers: shared hit=20
  ->  Index Scan using index_http_integrations_on_active_and_project_and_endpoint on public.alert_management_http_integrations  (cost=0.41..772.81 rows=1 width=161) (actual time=0.089..0.089 rows=1 loops=1)
        Output: id, created_at, updated_at, project_id, active, encrypted_token, encrypted_token_iv, endpoint_identifier, name
        Index Cond: ((alert_management_http_integrations.project_id = '4'::bigint) AND (alert_management_http_integrations.endpoint_identifier = '72e6ed9e7bf06d1b'::text))
        Buffers: shared hit=20
Planning Time: 0.049 ms
Execution Time: 0.100 ms

---

Related issue: #255513 (closed)

Screenshots

Does this MR meet the acceptance criteria?

Conformity

• [-] Changelog entry
• [-] Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team