Drop bridge jobs on unknown failures

What does this MR do and why?

Drop bridge jobs on unknown failures

Sometimes the underlying systems fail to respond and an error from them would leave the bridge in the running state undefinetly. Since we don't know the full list of errors that could cause this, we can catch everything and let the user decide if they want to retry or not.

Changelog: fixed

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After


How to set up and validate locally

1. Apply this patch to CreateDownstreamPipelineService to make the pipeline creation fail:

diff --git a/app/services/ci/create_downstream_pipeline_service.rb b/app/services/ci/create_downstream_pipeline_service.rb
index e38f5c988140..d9cbbed2d6c0 100644
--- a/app/services/ci/create_downstream_pipeline_service.rb
+++ b/app/services/ci/create_downstream_pipeline_service.rb
@@ -31,6 +31,8 @@ def execute(bridge)
 
       return ServiceResponse.error(message: 'Can not run the bridge') unless @bridge.run
 
+booooom
+
       service = ::Ci::CreatePipelineService.new(
         pipeline_params.fetch(:project),
         current_user,

2. Create a new pipeline that triggers a downstream pipeline:

test:
  script: echo 'tests'

deploy:
  trigger: 
    project: root/downstream-multiproject
    strategy: depend
  allow_failure: false

3. On master the deploy bridge job would remain in the running state and on this branch it will fail.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #423421 (closed)