Create pending escalations for incidents on update

What does this MR do and why?

• Related issues: #330284 (closed), #332065 (closed)
• Dependent on: !76819 (merged)

This MR:

• Creates pending escalations for incidents when relevant policy or status updates occur
• Moves helpers for "open" statuses from AlertManagement::Alert into Escalatable module
• Adds sidekiq processing for issue escalations following same flow as for PendingEscalation::Alert

Overall flow:

There are a number of steps to updates to IncidentManagement::IssuableEscalationStatus records. These steps roughly follow the flowchart below. The area circled in red is the scope of this MR.

How to set up and validate locally

1. Follow the steps in !76819 (merged)
2. Perform whatever updates to see how status/policy affect pending escalations. These are processed pretty quickly, so there's a bit of a time crunch to see the pending escalations in the DB 😅
   • For an incident w/o alerts:

     > # Set the status to resolved first just to make life easy; all escalations should be deleted
     > ::Issues::UpdateService.new(project: project, current_user: user, params: { escalation_status: { status: :resolved } }).execute(incident)
     => #<Issue id:673 root/autodevops#194>
     
     > # Set the policy to see the status retriggered & issue
     > ::Issues::UpdateService.new(project: project, current_user: user, params: { escalation_status: { policy: incident.project.incident_management_escalation_policies.first } }).execute(incident)
     => #<Issue id:673 root/autodevops#194>
     
     > ::IncidentManagement::PendingEscalations::Issue.count
     => 3
     
     > ::IncidentManagement::PendingEscalations::Issue.processable
     => [#<IncidentManagement::PendingEscalations::Issue:0x00007fd331e47a80
       id: 4,
       rule_id: 57,
       issue_id: 673,
       process_at: Thu, 16 Dec 2021 23:14:41.649844000 UTC +00:00,
       created_at: Thu, 16 Dec 2021 23:14:41.708339000 UTC +00:00,
       updated_at: Thu, 16 Dec 2021 23:14:41.708339000 UTC +00:00>,
      #<IncidentManagement::PendingEscalations::Issue:0x00007fd31bd189e0
       id: 5,
       rule_id: 59,
       issue_id: 673,
       process_at: Thu, 16 Dec 2021 23:14:41.649844000 UTC +00:00,
       created_at: Thu, 16 Dec 2021 23:14:41.740443000 UTC +00:00,
       updated_at: Thu, 16 Dec 2021 23:14:41.740443000 UTC +00:00>,
      #<IncidentManagement::PendingEscalations::Issue:0x00007fd31bd18918
       id: 6,
       rule_id: 60,
       issue_id: 673,
       process_at: Thu, 16 Dec 2021 23:14:41.649844000 UTC +00:00,
       created_at: Thu, 16 Dec 2021 23:14:41.772802000 UTC +00:00,
       updated_at: Thu, 16 Dec 2021 23:14:41.772802000 UTC +00:00>]

   • For an incident w/ an alert:

     > ::Issues::UpdateService.new(project: project, current_user: user, params: { escalation_status: { status: :resolved } }).execute(incident); ::Issues::UpdateService.new(project: project, current_user: user, params: { escalation_status: { status: :triggered } }).execute(incident)
     => #<Issue id:467 root/autodevops#9>
     
     > p ::IncidentManagement::PendingEscalations::Alert.count, ::IncidentManagement::PendingEscalations::Alert.processable
     => [3,
      [#<IncidentManagement::PendingEscalations::Alert:0x00007fd3218fff10
        id: 122,
        rule_id: 57,
        alert_id: 4,
        process_at: Thu, 16 Dec 2021 23:25:49.865673000 UTC +00:00,
        created_at: Thu, 16 Dec 2021 23:25:49.924825000 UTC +00:00,
        updated_at: Thu, 16 Dec 2021 23:25:49.924825000 UTC +00:00>,
       #<IncidentManagement::PendingEscalations::Alert:0x00007fd3218ffe48
        id: 123,
        rule_id: 59,
        alert_id: 4,
        process_at: Thu, 16 Dec 2021 23:25:49.865673000 UTC +00:00,
        created_at: Thu, 16 Dec 2021 23:25:49.954606000 UTC +00:00,
        updated_at: Thu, 16 Dec 2021 23:25:49.954606000 UTC +00:00>,
       #<IncidentManagement::PendingEscalations::Alert:0x00007fd3218ffd80
        id: 124,
        rule_id: 60,
        alert_id: 4,
        process_at: Thu, 16 Dec 2021 23:25:49.865673000 UTC +00:00,
        created_at: Thu, 16 Dec 2021 23:25:49.976096000 UTC +00:00,
        updated_at: Thu, 16 Dec 2021 23:25:49.976096000 UTC +00:00>]]

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.