Gitlab circuit breaker

What does this MR do and why?

Introduces Gitlab::CircuitBreaker which is a wrapper for https://github.com/yammer/circuitbox with certain defaults.

Circuitbox is a Ruby circuit breaker gem. It protects your application from failures of its service dependencies. It wraps calls to external services and monitors for failures in one minute intervals. Using a circuit's defaults once more than 5 requests have been made with a 50% failure rate, Circuitbox stops sending requests to that failing service for 90 seconds. This helps your application gracefully degrade.

This MR also moved the config for Circuitbox from EE to CE and switches LLM implementations of the circuit breaker to the Gitlab::CircuitBreaker.

The wrapper is called as:

Gitlab::CircuitBreaker.run_with_circuit('UniqueNamePerCircuit') do
  # do something
end

We set the following defaults but they can be passed as an options argument to the circuit:

• exceptions: [Gitlab::CircuitBreaker::InternalServerError]
• error_threshold: 50
• volume_threshold: 10
• sleep_window: 90 seconds (default set by gem)
• time_window: 60 seconds (default set by gem)

Gitlab::CircuitBreaker.run_with_circuit('UniqueNamePerCircuit', options = { volume_threshold: 2, error_threshold: 10 }) do
  # do something
end

How to set up and validate locally

1. Update lib/gitlab/circuit_breaker/notifier.rb line 9 to print out notifications, e.g. puts "notifying #{event} for circuit #{service_name}"
2. Start a rails console
3. Test a circuit without raising an error: Gitlab::CircuitBreaker.run_with_circuit('FirstCircuit') { puts 'Hello' } and note that it was a success from the printed output.
4. Trigger an error and note that it results in a failure notification: Gitlab::CircuitBreaker.run_with_circuit('FirstCircuit') { raise Gitlab::CircuitBreaker::InternalServerError }
5. Now let's force the circuit to be opened by setting the thresholds lower and triggering a few exceptions. Note that requests are skipped once the circuit is open.

   5.times { Gitlab::CircuitBreaker.run_with_circuit('SecondCircuit', options = { volume_threshold: 1 }) { raise Gitlab::CircuitBreaker::InternalServerError } }

6. Also note that SecondCircuit doesn't influence FirstCircuit: FirstCircuit is still closed and its thresholds are still the default even if SecondCircuit is open and has different thresholds.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #408207 (closed)