Custom email: Net::SMTP doesn't select the correct authentication method automatically

From the feedback issue:

Problem

I set up a Microsoft account myself and configured everything accordingly. It didn't work.

What's really interesting is, that it used to work, when I tried it before release. Maybe something in the configuration or the system itself changed 🤔

A short recap of how I configured it and what I tried

1. Enable Authenticated SMTP for the user in question (via the user flyout, Email and Apps section)
2. Enable 2FA for that account in Active Directory (now Entra).
3. Set Set-TransportConfig -SmtpClientAuthenticationDisabled $false via Azure Cloud Shell.
4. Also tried Set-TransportConfig -AllowLegacyTLSClients $true (reference)
5. Allow app password (via Entra admin center --> Named locations --> Configure MFA trusted IPs. (reference)
6. Sign in to the "custom email" account and configure 2FA (for example TOTP using an authenticator app) via Security Info.
7. Add an app password for that account on the same page. (reference)
8. I also configured a transport rule, but I didn't get so far, so that's optional for now.
9. Check credentials using swaks like mentioned in the comment above. It works!

   swaks --to user@example.com --from marc.saleiko@XYZ.onmicrosoft.com --auth-user marc.saleiko@XYZ.onmicrosoft.com --server smtp.office365.com:587 -tls-optional --auth-password superapppassword

   I used -tls-optional because that's closest to the setup we use internally.
10. Used the same credentials for the custom email configuration in my local environment. 500 error after a few seconds, because we don't catch the read timeout error yet.
11. I increased the read timeout locally and repeated the last step.
12. Now we get a different error 504 5.7.4 Unrecognized authentication type. Looks like Microsoft wants explicit TLS. Tried that, but it also didn't work.

Replication and testing

Feel free to use this sample script to run tests on your own:

smtp_options = {
  user_name: '',
  password: '',
  address: 'smtp.office365.com',
  port: 587,
  read_timeout: 7
}

recipient = 'name@gitlab.com'

mail = Mail.new do
  from    smtp_options[:user_name]
  to      recipient
  subject 'M365 test email'
  body    'test content'
end

mail.delivery_method(::Mail::SMTP, smtp_options)
mail.deliver

Basic Authenticated SMTP setup:

• Server: smtp.office365.com
• Port: 587 (recommended) or port 25
• TLS/StartTLS: Enabled
• Username/email address and password: Enter the sign-in credentials of the hosted mailbox being used

Solution path

• We require the SMTP server to return a list of AUTH types in the handshake phase. E.g. 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
• ActionMailer will then pick one method and send the request.
• If will fail with Net::SMTPAuthenticationError: 504 5.7.4 Unrecognized authentication type if no list is provided.
• Microsoft 365 doesn't provide such a list.
• Setting authentication: :login on the SMTP settings works, but needs to be set explicitly.
• So we probably need an additional setting where maintainers+ can set an authentication method explicitly or let the server decide which to choose.
• This will also be a great addition for customers of other service providers where the server also doesn't return a list or they want to use some challenge based AUTh method like cram_md5.