Add webhook delivery method options to config/mail_room.yml

What does this MR do and why?

For gitlab-com/gl-infra/scalability#1590

This MR is to add the following configurations to incoming_email and service_desk_email options in config/mail_room.yaml:

• delivery_method: those are two switches that force mail_room component to push email content to Rails web servers, instead of using Redis. They are the counterpart to the existing configuration in the helm chart.
• gitlab_url and gitlab_relative_url_root: there are the internal URL (properly workhorse URL) that mail_room will make request to. This config is identical to Gitlab Shell and Gitaly.

In a normal Rails application boot, the full GitLab URL can be generated via helpers like Gitlab.config.build_gitlab_url. In this case, mailroom renders config/mail_room.yml during boot. That file requires lib/gitlab/mail_room only. The rails environment is never loaded. We can't add rails-related stuff and heavy Gitlab configuration stack to mail_room either. So, the only reasonable approach is to pass gitlab urls from outside - Omnibus most of the time.

K8s helm chart doesn't use this file. It has its own version of the file, where the delivery url is set directly. config/mail_room.yml is generated from config/gitlab.yml. So, the configs are added to config/gitlab.yml.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

• Step 1. Upgrade mailroom gem to 0.0.19. For an historical reason, MailRoom version in the rails code base is still stuck at 0.0.9. There are plenty of bugs that make mailroom flow broken on local environment

diff --git a/Gemfile b/Gemfile
index 7a6e2a13c8c..538ae4c8a0d 100644
--- a/Gemfile
+++ b/Gemfile
@@ -441,7 +441,7 @@ gem 'octokit', '~> 4.15'

 # Updating this gem version here is deprecated. See:
 # https://docs.gitlab.com/ee/development/emails.html#mailroom-gem-updates
-gem 'gitlab-mail_room', '~> 0.0.9', require: 'mail_room'
+gem 'gitlab-mail_room', '~> 0.0.19', require: 'mail_room'

 gem 'email_reply_trimmer', '~> 0.1'
 gem 'html2text'

• Step 2: Generate a secret file for mailroom

echo "eao1Q7WZpH41L3tPdr0+NbUC6h8nHcQ9XE78mHxs5+A=" > /path-to-secret-location/.gitlab-mailroom-secret

• Step 3: set incoming_email and service_desk_email configuration, point to an email inbox.

  incoming_email:
    enabled: true
    address: "personal-email+%{key}@gmail.com"
    user: "personal-email@gmail.com"
    password: "something"
    delivery_method: webhook
    gitlab_url: http://127.0.0.1:3000
    secret_file: /path-to-secret-location/.gitlab-mailroom-secret

  service_desk_email:
    enabled: true
    address: "personal-email+%{key}@gmail.com"
    user: "personal-email@gmail.com"
    password: "something"
    delivery_method: webhook
    gitlab_url: http://127.0.0.1:3000
    secret_file: /path-to-secret-location/.gitlab-mailroom-secret

• Step 4: restart all services

• Step 5: start mailroom in a console with the following command

bundle exec mail_room -c ./config/mail_room.yml

• Step 6: tail the mailroom logs with the following command, assuming the recent working directory is GitLab rails.

tail -f ./log/mail_room_json.log

• Step 7: open a random project, configure Service Desk in that project:

https://gitlab.com/gitlab-org/gitlab/uploads/27075f3365cc0e469e64f738dea9a71f/Screen_Shot_2022-02-16_at_11.40.11.png

• Step 8: compose an email to send to the email address mentioned in the setting page

• Step 9: watch the mailroom logs, ensure that the email is handled. There should be a log file stating that the email content is delivered via webhook (postback - in mailroom terminology)

• Step 10: Verify that Rails web server receives a POST request sent to /api/v4/internal/mail_room/incoming_email

• Step 11: Verify that a job of EmailReceiverWorker is received, and handled by Sidekiq

• Step 12: Verify that there is an issue created in the service desk issue board

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #1590 (closed)