Automatic HTTPS certificate creation/renewal for Pages custom domains

Description

GitLab Pages allows users to specify custom domains to access their websites. Those websites could be accessible via HTTPS, but a certificate must be provided and renewed manually. Using Let's Encrypt, we can provide a simple way to obtain/renew the certificate and automatically configure it on Pages.

Note: Let's Encrypt support for the generic base domain is a separate issue discussed in omnibus-gitlab#3342.

Proposal

When a new custom domain is added and HTTPS is enabled, users can provide a certificate or choose to leverage Let's Encrypt to obtain one automatically. Certificates should be automatically managed for renewals. It's important it's on a per-domain basis because different users will have different domains with different requirements.

Users should be able to configure Let's Encrypt in the Application settings by filling in an email for alerts, and accepting Let's Encrypt terms of service. Once configured and enabled, users will have the option to toggle Let's Encrypt on/off on project Pages settings.

Users that have no access permission to Let's Encrypt should not be able to manage it.

UX Proposal

NOTE: In case the copy in the prototypes is different from the one in the acceptance criteria below, please consider the written acceptance criteria as the SSO and not the prototypes.

Requirement	Acceptance criteria	Prototype
As an admin, I want to configure Let's Encrypt	- In the Application settings > preferences, Pages section, user sees an option to configure Let's Encrypt. - The helper text should read: Let's Encrypt is a free, automated, and open certificate authority (CA), that give digital certificates in order to enable HTTPS (SSL/TLS) for websites. Learn more about [SSL Configuration in the documentation]. - An input text field is enabled for email, and a checkbox should be available for Let's Encrypt terms of service. - User needs to enter a valid email and check the Terms of Service checkbox in order to configure Let's Encrypt. - An error state should be enabled in the form after the user clicks the 'Save' button without entering a valid email or checking the checkbox. The email field should not be validated if: the user has not selected the checkbox (no changes in the form).
As a user, I want to use Let's Encrypt when configured	- If Let's Encrypt is configured by the admin, the user should see a new option in the project Pages settings. A toggle button should be displayed allowing the user toggle Let's Encrypt on/off for Pages. - The default is "Off" - A helper text should be added to help users understand what Let's Encrypt can do, and link to the configuration page in the Docs. - Toggling Let's Encrypt ON should hide the 'Key (PEM) text area. While the Let's Encrypt certificate is being generated, user sees an empty state message where the 'Certificate (PEM)' field is. Once the certificate is generated, user should see the Let's Encrypt certificate in a textarea. - Toggling Let's Encrypt OFF should display the 'Key (PEM)' text area.
As a user, I want to use a certificate when Let's Encrypted is not configured	- If Let's Encrypt is not configured by the Application's admin, users should not see the toggle to option Let's Encrypt on/off in the Pages configuration screen.

Other notes:

• To reduce security risks we should never make the let's encrypt private key visible to any users at any time since they do not need to know it. Right now it would appear that private keys for certificates are visible when editing a pages domain. This should not be visible for let's encrypt certificates, so we should not give users ability to see uploaded key on domain edit page.
• Enabling it for the first time may take a bit of time. We need to show a warning that the certificate is being fetched and the site won't work until that is done (~15m)
• If the user deselects automatic configuration, everything should turn back to the way it was and we should release the certificate with LetsEncrypt.
• Add a helper text next to the certificates section with some information regarding what they are and what they do, and link it to: https://gitlab.com/help/user/project/pages/getting_started_part_three.md#adding-certificates-to-your-project. This way we provide more context about the certificates. It should read: Learn more about adding certificates to your project following the [documentation on GitLab pages].

Implementation Plan

gitlab-pages gitlab-pages#165 (closed)

• Changes so that it redirects any requests for unknown files matching the acme verification format:

GET mydomain.com/.well-known/acme-challenge/:acme_key

=>

301
Location: gitlab.com/-/acme-challenge/mydomain.com/:acme_key

gitlab-ce

• AcmeChallengeController:
  • Looks up PagesDomain#acme_key, PagesDomain#acme_account_key and serves up the correct expected ACME response.
  • Importantly this should not serve up the certificate unless PagesDomain#verified? to avoid any risk of us serving up SSL certificates for domains people do not own.
• PagesDomainSslCertificateWorker
  • uses the ACME API to create PagesDomain#acme_account_key unless it exists.
  • uses the ACME API to generate a PagesDomain#acme_key and persists this to the DB so it can be used by AcmeChallengeController.
  • requests ACME API to now validate the domain and return certificate
  • persists certificate to PagesDomain#certificate and persists private key details
  • persists PagesDomain#certificate_expires_at to be 90 days from now (or whatever ACME API returns)
  • kicks off Projects::UpdatePagesConfigurationService
• PagesDomainSslCertificateCronWorker which runs periodically to check for all PagesDomain.certificate_expires_at within the next 10 days and kicks off PagesDomainSslCertificateWorker fork those.

Security Notes

• We really want to make sure there is no way people can exploit this to get SSL certificates for someone else's domain
• Probably to reduce security risks we should never make the let's encrypt data and private key visible to any users at any time since they do not need to know it. Right now it would appear that private keys for certificates are visible when editing a pages domain. This should probably not be visible for let's encrypt certificates. In fact it's probably better it's not visible for any certificates.

What does success look like, and how can we measure that?

Measure users with this feature turned on

Unfinished parts

1. gitlab-pages!141 (merged) - gitlab-pages go part, ready to be merged, waits for following MR to omnibus to be ready
2. omnibus-gitlab!3314 (merged) - omnibus support for new gitlab-pages config params in dev
3. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26438 - frontend part, in review, should be merged soon
4. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27811/diffs - main CE part, in review
5. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28855 - fix of already merged parts, in review
6. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28743 - store valid period for ssl cert, in review
7. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29359 - fixes follow ups(https://gitlab.com/gitlab-org/gitlab-ce/issues/61752, https://gitlab.com/gitlab-org/gitlab-ce/issues/61500, https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27947) in review
8. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29363 - scheduling obtaining/renewal of certificates, in review
9. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29609 - make related feature flag "aware of project" - should be small, but would be easier to do when previous 2 MR got merged, in review
10. do not get certs for not verified domains - in review(also in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29609)
11. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29675 - reduce waiting time(automatically schedule subsequent stages of obtaining certificate) - in review
12. actually rollout changes on production (may be we need a separate issue here)
13. write documentation - not started, this can be done simultaneously with production rollout

points for future documentation

1. If a certificate is obtained through LetsEncrypt and then LetsEncrypt integration is disabled in GitLab admin settings, the obtained certificate will continue to work until replaced manually.