Skip to content

Set webhook as mailroom default delivery method

Quang-Minh Nguyen requested to merge qmnguyen0711/set-webhook-as-mailroom-default-delivery-method into master

What does this MR do?

For gitlab-com/gl-infra/scalability#1462 (closed)

In !5927 (merged), we introduced a new deliveryMethod option of Gitlab. This component receives incoming email and service email, then delivers the email content to Rails web service. The deliveryMethod allows the customers to choose between two supported methods: webhook and sidekiq. GitLab.com has switched to use webhook delivery method for a while. I think it's time to switch the default method to webhook for all self-managed instances.

Testing scenarios

This MR does not introduce a feature change. It sets a default configuration, and contains some refactorings. Therefore, the following scenarios focus on testing the generation of gitlab.yml. Functionality test can be found here: !5927 (merged).

For each case, I updated /etc/gitlab/gitlab.rb in a virtual machine, ran gitlab-ctl reconfigure, and compared the result of generated gitlab.yml file.

Case 1: Minimal configuration, delivery method is not configured

/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.external.url'

gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "gitlab@gmail.com"
gitlab_rails['incoming_email_password'] = "[REDACTED]"
# incoming_email_delivery_method is not set

gitlab_rails['service_desk_email_enabled'] = true
gitlab_rails['service_desk_email_address'] = "contact_project+%{key}@gmail.com"
gitlab_rails['service_desk_email_email'] = "contact_project@gmail.com"
gitlab_rails['service_desk_email_password'] = "[REDACTED]"
# service_desk_email_delivery_method is not set
/opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml 
  incoming_email:
    enabled: true
    address: "gitlab+%{key}@gmail.com"
    user: "gitlab@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.external.url
    secret_file: .gitlab_incoming_email_secret

  service_desk_email:
    enabled: true
    address: "contact_project+%{key}@gmail.com"
    user: "contact_project@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.external.url
    secret_file: .gitlab_service_desk_email_secret

Case 2: internal_api_url is set

/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.external.url'
gitlab_rails['internal_api_url'] = 'http://gitlab.internal.api.url'

gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "gitlab@gmail.com"
gitlab_rails['incoming_email_password'] = "[REDACTED]"
# incoming_email_delivery_method is not set

gitlab_rails['service_desk_email_enabled'] = true
gitlab_rails['service_desk_email_address'] = "contact_project+%{key}@gmail.com"
gitlab_rails['service_desk_email_email'] = "contact_project@gmail.com"
gitlab_rails['service_desk_email_password'] = "[REDACTED]"
# service_desk_email_delivery_method is not set
/opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml 
  incoming_email:
    enabled: true
    address: "gitlab+%{key}@gmail.com"
    user: "gitlab@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.internal.api.url # 👈
    secret_file: .gitlab_incoming_email_secret

  service_desk_email:
    enabled: true
    address: "contact_project+%{key}@gmail.com"
    user: "contact_project@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.internal.api.url # 👈
    secret_file: .gitlab_service_desk_email_secret

Case 3: workhorse listen_addr is configured

/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.external.url'
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_addr'] = "127.0.0.1:8181"

gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "gitlab@gmail.com"
gitlab_rails['incoming_email_password'] = "[REDACTED]"
# incoming_email_delivery_method is not set

gitlab_rails['service_desk_email_enabled'] = true
gitlab_rails['service_desk_email_address'] = "contact_project+%{key}@gmail.com"
gitlab_rails['service_desk_email_email'] = "contact_project@gmail.com"
gitlab_rails['service_desk_email_password'] = "[REDACTED]"
# service_desk_email_delivery_method is not set
/opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml 
  incoming_email:
    enabled: true
    address: "gitlab+%{key}@gmail.com"
    user: "gitlab@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://127.0.0.1:8181 # 👈
    secret_file: .gitlab_incoming_email_secret

  service_desk_email:
    enabled: true
    address: "contact_project+%{key}@gmail.com"
    user: "contact_project@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://127.0.0.1:8181 # 👈
    secret_file: .gitlab_service_desk_email_secret

Case 4: workhorse listen_addr is configured with Unix socket

/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.external.url'
gitlab_workhorse['listen_network'] = "unix"
gitlab_workhorse['listen_addr'] = "/var/opt/gitlab/gitlab-workhorse/sockets/socket"

gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "gitlab@gmail.com"
gitlab_rails['incoming_email_password'] = "[REDACTED]"
# incoming_email_delivery_method is not set

gitlab_rails['service_desk_email_enabled'] = true
gitlab_rails['service_desk_email_address'] = "contact_project+%{key}@gmail.com"
gitlab_rails['service_desk_email_email'] = "contact_project@gmail.com"
gitlab_rails['service_desk_email_password'] = "[REDACTED]"
# service_desk_email_delivery_method is not set
/opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml 
  incoming_email:
    enabled: true
    address: "gitlab+%{key}@gmail.com"
    user: "gitlab@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.external.url # 👈 Fallback to external url
    secret_file: .gitlab_incoming_email_secret

  service_desk_email:
    enabled: true
    address: "contact_project+%{key}@gmail.com"
    user: "contact_project@gmail.com"
    password: "[REDACTED]"
    delivery_method: webhook # 👈
    gitlab_url: http://gitlab.external.url # 👈 Fallback to external url
    secret_file: .gitlab_service_desk_email_secret

Case 5: incoming mail delivery method is set to Sidekiq

/etc/gitlab/gitlab.rb 
external_url 'http://gitlab.external.url'
gitlab_workhorse['listen_network'] = "unix"
gitlab_workhorse['listen_addr'] = "/var/opt/gitlab/gitlab-workhorse/sockets/socket"

gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "gitlab@gmail.com"
gitlab_rails['incoming_email_password'] = "[REDACTED]"
gitlab_rails['incoming_email_delivery_method'] = "sidekiq"

gitlab_rails['service_desk_email_enabled'] = true
gitlab_rails['service_desk_email_address'] = "contact_project+%{key}@gmail.com"
gitlab_rails['service_desk_email_email'] = "contact_project@gmail.com"
gitlab_rails['service_desk_email_password'] = "[REDACTED]"
gitlab_rails['service_desk_email_delivery_method'] = "sidekiq"
/opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml 
  incoming_email:
    enabled: true
    address: "gitlab+%{key}@gmail.com"
    user: "gitlab@gmail.com"
    password: "[REDACTED]"
    delivery_method: sidekiq

  service_desk_email:
    enabled: true
    address: "contact_project+%{key}@gmail.com"
    user: "contact_project@gmail.com"
    password: "[REDACTED]"
    delivery_method: sidekiq

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • Pipeline is green on dev.gitlab.org if the change is touching anything besides documentation or internal cookbooks
  • trigger-package has a green pipeline running against latest commit

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for the GitLab Chart opened
Edited by Quang-Minh Nguyen

Merge request reports