Add worker for marking pending compliance external controls as failed

What does this MR do and why?

1. This MR is forked from !180010 (merged) and should only be merged once we merge that MR.
2. This MR creates a worker to mark compliance external controls as failed if they are in a pending state since more than 30 minutes.
3. The worker is invoked after 30 minutes of sending the current project payload to the user's external control endpoint.
4. We would create an API in this issue that will allow user's to mark the pending external controls as fail or pass.
5. However, if the user doesn't provide the status within 30 minutes then the above worker would mark it as fail and also create an audit event reflecting the same.

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Ensure you have ultimate license.
2. For verification you need to create a compliance framework.
3. You would also need to create at least one compliance requirement for this framework using the GraphQL API.
4. You would also need to create at least one external control for the above compliance requirement. We do not support GraphQL API for this so you can use rails console for now.

cr = ComplianceManagement::ComplianceFramework::ComplianceRequirement.last
cr.compliance_requirements_controls.create!(control_type: 'external', name: "external_control", namespace_id: cr.namespace_id, external_url: "https://huzaifaiftikhar.requestcatcher.com", secret_token: "SECRET")

5. Since this service is not exposed via an API or UI, we would need to use rails console for validation.

project = Project.find(20) # or any other existing project ID

framework = ComplianceManagement::Framework.last

cr = framework.compliance_requirements.last

control = cr.compliance_requirements_controls.last

ComplianceManagement::ComplianceFramework::ComplianceRequirements::TriggerExternalControlService.new(project, control).execute

6. Check that the status of ProjectControlComplianceStatus is marked as pending.

ComplianceManagement::ComplianceFramework::ProjectControlComplianceStatus.for_project_and_control(project.id, control.id)

7. After 30 mins again check the the status and now it should be marked as fail.
8. For testing we can reduce the delay time by applying the below patch

Diff

diff --git a/ee/app/services/compliance_management/compliance_framework/compliance_requirements/trigger_external_control_service.rb b/ee/app/services/compliance_management/compliance_framework/compliance_requirements/trigger_external_control_service.rb
index 23cbca83fdd9..b2910e334066 100644
--- a/ee/app/services/compliance_management/compliance_framework/compliance_requirements/trigger_external_control_service.rb
+++ b/ee/app/services/compliance_management/compliance_framework/compliance_requirements/trigger_external_control_service.rb
@@ -69,7 +69,7 @@ def handle_response(response)
           if response.success?
             audit_success(response.code)

-            ComplianceManagement::TimeoutPendingExternalControlsWorker.perform_in(31.minutes,
+            ComplianceManagement::TimeoutPendingExternalControlsWorker.perform_in(1.minute,
               { 'control_id' => control.id, 'project_id' => project.id })

             ServiceResponse.success(payload: { control: control })
diff --git a/ee/app/workers/compliance_management/timeout_pending_external_controls_worker.rb b/ee/app/workers/compliance_management/timeout_pending_external_controls_worker.rb
index 97e06dbdb574..8d0d6844b9b6 100644
--- a/ee/app/workers/compliance_management/timeout_pending_external_controls_worker.rb
+++ b/ee/app/workers/compliance_management/timeout_pending_external_controls_worker.rb
@@ -21,7 +21,7 @@ def perform(args = {})
                                             .for_project_and_control(project_id, control_id).last

       unless project_control_compliance_status&.pending? &&
-          project_control_compliance_status.updated_at < 30.minutes.ago
+          project_control_compliance_status.updated_at < 50.seconds.ago
         return
       end

Closes #513423 (closed)

assigned to @huzaifaiftikhar1

added backend featureaddition groupcompliance labels

added devopssoftware supply chain security sectionsec typefeature labels

changed milestone to %17.9

added pipelinetier-3 pipeline:run-e2e-omnibus-once labels

added 2 commits

• b3c2c8aa - 1 commit from branch 513421_create_service_for_external_controls
• d256257d - Add worker to timeout pending compliance external controls

Compare with previous version

added 1 commit

• 65325197 - Add worker to timeout pending compliance external controls

Compare with previous version

added databasereview pending documentation labels

	1 Message
	This merge request adds or changes documentation files and requires Technical Writing review. The review should happen before merge, but can be post-merge if the merge request is time sensitive.

Documentation review

The following files require a review from a technical writer:

• doc/user/compliance/audit_event_types.md (Link to current live version)

The review does not need to block merging this merge request. See the:

• Metadata for the *.md files that you've changed. The first few lines of each *.md file identify the stage and group most closely associated with your docs change.
• The Technical Writer assigned for that stage and group.
• Documentation workflows for information on when to assign a merge request for review.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@gonzoyumo (UTC-5, 10.5 hours behind author)	@kerrizor (UTC-8, 13.5 hours behind author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

Sidekiq queue changes

This merge request contains changes to Sidekiq queues. Please follow the documentation on changing a queue's urgency.

These queues were added:

• compliance_management_timeout_pending_external_controls

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

changed this line in version 5 of the diff

changed this line in version 5 of the diff

added 2 commits

• 4627161d - 1 commit from branch 513421_create_service_for_external_controls
• d8c1f0b0 - Add worker to timeout pending compliance external controls

Compare with previous version

mentioned in issue #513423 (closed)

changed the description

removed databasereview pending label

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-cng: test report for fffea965

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Create      | 140    | 0      | 22      | 0     | 162   | ✅     |
| Govern      | 83     | 0      | 11      | 0     | 94    | ✅     |
| Package     | 29     | 0      | 15      | 0     | 44    | ✅     |
| Plan        | 86     | 0      | 8       | 0     | 94    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Secure      | 2      | 0      | 5       | 0     | 7     | ✅     |
| Verify      | 52     | 0      | 20      | 0     | 72    | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 443    | 0      | 128     | 0     | 571   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-gdk: test report for fffea965

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
| Verify      | 51     | 0      | 21      | 0     | 72    | ✅     |
| Create      | 135    | 0      | 23      | 0     | 158   | ✅     |
| Plan        | 82     | 0      | 8       | 0     | 90    | ✅     |
| Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
| Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
| Govern      | 78     | 0      | 14      | 0     | 92    | ✅     |
| Package     | 24     | 0      | 14      | 0     | 38    | ✅     |
| Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
| Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
| Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
| Secure      | 4      | 0      | 3       | 0     | 7     | ✅     |
| Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
| Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 425    | 0      | 130     | 0     | 555   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

added 4 commits

• d8c1f0b0...f8c3c7f0 - 3 commits from branch 513421_create_service_for_external_controls
• 2a3450e7 - Add worker to timeout pending compliance external controls

Compare with previous version

added 2 commits

• 4eb20c75 - 1 commit from branch 513421_create_service_for_external_controls
• c06a3fc9 - Add worker to timeout pending compliance external controls

Compare with previous version

deleted the 513421_create_service_for_external_controls branch. This merge request now targets the master branch

marked this merge request as ready