Draft: Parallelize CI remote includes with threads

What does this MR do and why?

Related to #351250 (closed)

CI remote includes can take seconds and we don't need to wait for calls to complete, we can jump to the next include. In this MR, we are using Concurrent::Promise to parallelize the HTTP calls of CI remote includes.

I used the already existing gem concurrent-ruby in this. I added it to Gemfile. The gem is thread-safe, well-maintained, and updated frequently. Doc: https://ruby-concurrency.github.io/concurrent-ruby/1.2.0/Concurrent/Promise.html

These changes are behind the FF ci_parallel_remote_includes (#413770 (closed))

p.s. we tried to use Concurrent::Promise before in !71549 (closed) in a different way.

How to set up and validate locally

1. Enable the feature flag;

   Feature.enable(:ci_parallel_remote_includes)

2. Run a basic web server to test slow includes.

   require 'rack'
   
   def run(env)
     puts "URL: #{env['REQUEST_URI']} started"
     sleep 3
     puts "URL: #{env['REQUEST_URI']} finished"
     "
     test:
       script: echo 'hello world'
     "
   end
   
   app = Proc.new do |env|
     [
       200,
       { "Content-Type" => "text/x-yaml" },
       [run(env)]
     ]
   end
   
   Rack::Server.start(app: app, Port: 9292)

3. Enable local includes on the Admin page http://gdk.test:3000/admin/application_settings/network
4. Use a basic CI config;

   include:
     - remote: http://0.0.0.0:9292/file1.yml
     - remote: http://0.0.0.0:9292/file2.yml
     - remote: http://0.0.0.0:9292/file3.yml
     - remote: http://0.0.0.0:9292/file4.yml
     - remote: http://0.0.0.0:9292/file5.yml
   
   build:
     script: exit 0

5. Result;

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.