Added headers hash for http destinations

What does this MR do and why?

This MR adds a method for us to return headers_hash from any model that composes the ExternallyStreamable concern. It overwrites any attempt to write over X-Gitlab-Event-Streaming-Token which is set to the secret_token (generated or user provided)

This MR also updates the json schema file for HTTP categories, and uses the HTTP RFC-7230 to allow certain characters.

• It is pretty permissive on what is allowed (USASCII, special characters, spaces, etc.)
• An alternative is to use what Cloudflare defines as acceptable header keys and values format: Reference
• It also disallows any 'custom' properties, so only value and active are passable

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Checkout the git branch
2. Create new external streaming destinations using graphql with the category http
3. Open rails console and query for streaming destinations [AuditEvents::Instance::ExternalStreamingDestination]
4. Validate that headers_hash is returned for http category destinations

Example graphql query

mutation CreateInstanceAuditEventStreamingDestination {
  instanceAuditEventStreamingDestinationsCreate(
    input: {
      config: {
        url:"https://gitlab.com/test",
        headers: { key1: { value: "test", active: true } }
      },
      category: "http", 
      secretToken:"my-token-my-token-12345",
    }
  ) {
    externalAuditEventDestination {
      id
      name
      category
      config
    }
    errors
  }
}

Related to #436607 (closed)

changed milestone to %17.4

added Deliverable automation:ml backend devopsgovern groupcompliance missed-deliverable missed:16.9 missed:17.1 priority2 sectionsec typefeature workflowin dev labels

assigned to @hraghuvanshi

added pipelinetier-1 label

	2 Warnings
	This MR changes code in ee/, but its Changelog commit is missing the EE: true trailer. Consider adding it to your Changelog commits.
	b4cf955d: The commit body should not contain more than 72 characters per line. For more information, take a look at our Commit message guidelines.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@syasonik (UTC-5, 10.5 hours behind author)	@mkozono (UTC-10, 15.5 hours behind author)
groupcompliance	Reviewer review is optional for groupcompliance	@nradina (UTC+1, 4.5 hours behind author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

changed milestone to %17.5

added missed:17.4 label

mentioned in epic gitlab-org#12339

mentioned in issue #436607 (closed)

changed milestone to %17.6

added missed:17.5 label

assigned to @andrew.jung

added 7942 commits

• fbe58a5b...73407730 - 7941 commits from branch master
• 028552a3 - Merge remote-tracking branch 'origin/master' into 436607-http-headers

Compare with previous version

added 1 commit

• d8f8656b - Overwrite any changes to our header to normal, add better regex for headers, update spec tests

Compare with previous version

unassigned @hraghuvanshi

marked this merge request as ready

added 734 commits

• d8f8656b...f0e92b9e - 733 commits from branch master
• 4c27d280 - Added headers hash for http destinations

Compare with previous version

changed the description

Pinging @Quintasan for backend review and @hraghuvanshi for maintainer / author review

requested review from @hraghuvanshi and @Quintasan

approved this merge request

added pipeline:mr-approved label

added pipelinetier-2 label and removed pipelinetier-1 label

Before you set this MR to auto-merge

This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.

Before you set this MR to auto-merge, please check the following:

• You are the last maintainer of this merge request
• The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
• This pipeline is recent enough (created in the last 8 hours)

If all the criteria above apply, please set auto-merge for this merge request.

See pipeline tiers and merging a merge request for more details.

Generated by gitlab_quality-test_tooling.

---

Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.

Click to expand

Job	File	Name	Duration	Expected duration
#8163335417	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	82.63 s	< 27.12 s
#8176104629	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	78.11 s	< 27.12 s
#8178116979	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	90.78 s	< 27.12 s
#8181075891	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	77.87 s	< 27.12 s
#8276900678	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	78.12 s	< 27.12 s
#8277884525	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	83.49 s	< 27.12 s
#8278349671	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	87.77 s	< 27.12 s
#8295396781	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	79.72 s	< 27.12 s

added rspec:slow test detected label

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-gdk: test report for b4cf955d

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Create      | 8      | 0      | 0       | 0     | 8     | ✅     |
| Package     | 1      | 0      | 0       | 0     | 1     | ✅     |
| Govern      | 4      | 0      | 0       | 0     | 4     | ✅     |
| Data Stores | 2      | 0      | 0       | 0     | 2     | ✅     |
| Verify      | 1      | 0      | 0       | 0     | 1     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 16     | 0      | 0       | 0     | 16    | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-cng: test report for b4cf955d

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Plan        | 86     | 0      | 8       | 0     | 94    | ✅     |
| Create      | 139    | 0      | 21      | 0     | 160   | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Verify      | 50     | 0      | 15      | 0     | 65    | ✅     |
| Govern      | 83     | 1      | 9       | 3     | 93    | ❌     |
| Secure      | 1      | 0      | 6       | 0     | 7     | ✅     |
| Package     | 24     | 0      | 14      | 0     | 38    | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 434    | 1      | 120     | 3     | 555   | ❌     |
+-------------+--------+--------+---------+-------+-------+--------+

added workflowready for review label and removed workflowin dev label

resolved all threads

added 544 commits

• 4c27d280...a54d998d - 543 commits from branch master
• 378e73df - Added headers hash for http destinations

Compare with previous version

added 1 commit

• 5eead437 - Clean up headers_hash filter

Compare with previous version

reset approvals from @Quintasan by pushing to the branch

added 31 commits

• 5eead437...b0781539 - 30 commits from branch master
• 8e1d8fbf - Added headers hash for http destinations

Compare with previous version

resolved all threads

added 96 commits

• 8e1d8fbf...4be2afdc - 95 commits from branch master
• 3779ad85 - Added headers hash for http destinations

Compare with previous version

@Quintasan Could you review again for backend as I added a small change to headers_hash and rebased,

approved this merge request

enabled automatic add to merge train when checks pass

resolved all threads

@hraghuvanshi could you do maintainer review/approval -- thank you

requested review from @alipniagov

approved this merge request

resolved all threads

removed review request for @alipniagov