Adds Service Desk custom email verification services
Feature context
Click to expand 👇
Right now it is not possible to customize the Service Desk email address (intake and sending) in its entirety. On self-hosted instances you have more control over the used addresses, but you will still have a rather cryptic target email address for a specific service desk in a project. For .com users it's currently not possible to customize the Service Desk email at all.
There is a proposal and a further exploration around this issue. A summary of the solution path is the following: Users set up their custom email to forward all emails to the cryptic Service Desk email and provide SMTP credentials so we can send emails on their behalf. This way customers seeking support will only see the custom email address in their communication.
There is further discussion about improving and changing the general infrastructure, but this approach is a MVC to solve the issue for our customers.
🗺 How does it contribute to the whole feature?
This MR is the second part in a series of MRs that will follow in order to complete this feature. See #329990 (comment 1227384943) for a detailed breakdown. Here's a summary:
-
✅ Using SMTP credentials. Foundation work. Add Service Desk custom email foundation (!108017 - merged) -
🎯 Verify email ownership, correct function and setup Part 8: Add verification services - Ingest replies from custom email
- Add settings and validation to Settings page
- Add documentation
What does this MR do and why?
This MR adds the logic for the verification part of Customizable email addresses for Service Desk. The ServiceDesk::CustomEmailVerifications::CreateService
starts the verification process and tries to send the verification email and ServiceDesk::CustomEmailVerifications::UpdateService
checks the incoming verification email and moves the verification entry to finished
or failed
.
We can only call the services from the console right now. The email ingestion (***EmailHandler
is not connected in this MR yet).
CreateService
Flow for -
CreateService
starts the verification process, createsServiceDesk::CustomEmailVerification
instarted
state and tries to send a verification email (directly) with the provided settings inServiceDesk::CustomEmailCredential
. - We notify project owners and the user who triggered the verification.
- If an error occurred while sending the email, we'll end the verification process, mark it as failed. We notify project owners and the user about the failure.
- If the email was sent, we are done here.
flowchart TD
A[Start] -->B{CreateService}
B -->|success| C[Sent verification email]
B -->|failure| D[End process, send failure email]
UpdateService
Flow for -
UpdateService
takes an email as additional parameter. - If the verification email was ingested successfully, we have the email object as a parameter and check for a few things
- Was the email received within the 30 minutes timeframe?
- Does the
From
header match the original sender (this is us. Some service providers do not preserve the original from. We need that for correct author attribution) - Is the verification token correct?
- Only if all the above was successful, we mark the verification as finished and move it to the
finished
state. If not we move it tofailed
.
- We also notify project owners and the user who triggered the verification.
- If we call the
UpdateService
and we have already verified it, we exit early. - If we call the
UpdateService
and it has already failed, we exit early.
flowchart TD
A(Ingested verification email) --> B
D(Without email) --> B{UpdateService}
B -->|mark as finished| E(verified)
B -->|mark as failed| F(verification failed)
Screenshots or screen recordings
How to set up and validate locally
mail_room
, instead we use prepared email eml
files.
-
Find a project ID in your installation that you have not used for any Service Desk (including
CustomEmailCredentials
andCustomEmailVerification
testing) setup and testing. Why? It makes these steps a lot easier🙂 -
Open the rails console
bin/rails c
ingitlab
folder -
Find the project by id
project = Project.find(5) # Where 5 is your project id
-
Get the root user (as it's the owner of each project)
user = User.first
-
Create
ServiceDeskSetting
entry (which holds Service Desk related meta info and custom email)settings = ServiceDeskSetting.create!( project: project, custom_email: 'user@gmail.com' # Use your "real" test email address )
-
Create
ServiceDesk::CustomEmailCredential
# Use your "real" test email credentials here. If you use a GMail address # you only need to change the smtp_username and smtp_password credential = ServiceDesk::CustomEmailCredential.create!( project: project, smtp_address: 'smtp.gmail.com', smtp_port: 587, smtp_username: 'user@gmail.com', smtp_password: 'supersecret' )
-
Reset associations on
project
, because we access these objects fromServiceDeskSetting
internally viaproject
project.reset
-
Call the
CreateService
. It should return aServiceResponse
withsuccess
. You should see a notification email in Letter Opener and in your custom email mailbox you should find the verification email.ServiceDesk::CustomEmailVerifications::CreateService.new(project: project, current_user: user).execute
-
You can try out error cases, if you change your credentials. In those cases you should not see the verification email in your custom email mailbox, but see two notifications in Letter Opener (1. verification has been triggered and 2. result of the verification --> error)
- Change the
smtp_host
to something likewrong.gmail.com
(error
smtp_host_issue
) - Change your password to something wrong etc.
- Change the
-
Now we move to
UpdateService
: For this to work, we need aMail
object with a verification email. Let's take the example from the specs and modify it, so that it contains our defined custom email address:verification_email_text = <<~EMAIL Delivered-To: support+project_slug-project_id-issue-@example.com Received: by 2002:a05:7022:aa3:b0:5d:66:2e64 with SMTP id dd35csp3394266dlb; Mon, 23 Jan 2023 08:50:49 -0800 (PST) X-Received: by 2002:a19:a40e:0:b0:4c8:d65:da81 with SMTP id q14-20020a19a40e000000b004c80d65da81mr9022372lfc.60.1674492649184; Mon, 23 Jan 2023 08:50:49 -0800 (PST) Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id t20-20020a195f14000000b00499004f4b1asor10121263lfb.188.2023.01.23.08.50.48 for <support+project_slug-project_id-issue-@example.com> (Google Transport Security); Mon, 23 Jan 2023 08:50:49 -0800 (PST) X-Received: by 2002:a05:6512:224c:b0:4cc:7937:fa04 with SMTP id i12-20020a056512224c00b004cc7937fa04mr1421048lfu.378.1674492648772; Mon, 23 Jan 2023 08:50:48 -0800 (PST) X-Forwarded-To: support+project_slug-project_id-issue-@example.com X-Forwarded-For: custom-support-email@example.com support+project_slug-project_id-issue-@example.com Return-Path: <custom-support-email@example.com> Received: from gmail.com ([94.31.107.53]) by smtp.gmail.com with ESMTPSA id t13-20020a1c770d000000b003db0ee277b2sm11097876wmi.5.2023.01.23.08.50.47 for <fatjuiceofficial+verify@gmail.com> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Jan 2023 08:50:47 -0800 (PST) From: Flight Support <custom-support-email@example.com> X-Google-Original-From: Flight Support <example@example.com> Date: Mon, 23 Jan 2023 17:50:46 +0100 Reply-To: GitLab <noreply@example.com> To: custom-support-email+verify@example.com Message-ID: <63d927a0e407c_5f8f3ac0267d@mail.gmail.com> Subject: Verify custom email address custom-support-email@example.com for Flight Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Auto-Submitted: no X-Auto-Response-Suppress: All This email will verify the ownership of the enered custom email address and correct functionality of the email forwarder. Verification token: ZROT4ZZXA-Y6 -- You're receiving this email because of your account on 127.0.0.1. EMAIL # Add our custom email here verification_email_text.gsub!('custom-support-email@example.com', settings.custom_email) verification_email_text.gsub!('custom-support-email+verify@example.com', settings.custom_email_address_for_verification) # Make a Mail object from the text verification_mail = Mail.new(verification_email_text)
-
Now let's update the token of the verification object and use the token from the email:
verification = project.service_desk_custom_email_verification verification.update!(token: 'ZROT4ZZXA-Y6') project.reset
-
Run the
UpdateService
. and provide the mail object as param. You should see a notification email that says that the email address has been verified in Letter Opener. Additionally the service should return aServicResponse
withsuccess
ServiceDesk::CustomEmailVerifications::UpdateService.new(project: project, params: { mail: verification_mail }).execute
-
Now you can play a bit with this to force different error scenarios:
-
For a fresh start you can always reset the verification to
started
withverification.mark_as_started!(user) project.reset
-
Provide no email would lead to
mail_not_received_within_timeframe
error witherror
ServiceResponse
-
Provide a
ramp_up_error
e.g. likeparams: { ramp_up_error: 'smtp_host_issue' }
, which would lead tosmtp_host_issue
error witherror
ServiceResponse
-
Move
triggered_at
back in time and call the service like in the success example. This would lead tomail_not_received_within_timeframe
error witherror
ServiceResponse
verification.mark_as_started!(user) verification.update!(triggered_at: 40.minutes.ago) project.reset
-
Provide an invalid token or modify the
From
header to something else to see the corresponding errors. -
Call the service twice to see that it exits early at the second time without writing to the database
-
-
(Optional) clean up the mess and remove entries
settings.destroy credential.destroy verification.destroy
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.