Pages domain verification: infer ownership of subdomains

In GitLab 10.5, we added pages domain verification. It would be good to enhance this so that if a project has proven ownership of domain.com, a custom domain like sub.domain.com will also be automatically verified without needing to add many additional TXT records.

There are potential security concerns with this, however - I don't think it's always the case that someone who can publish records for domain.com can also control sub.domain.com. Needs more research.

/cc @Jonarod @jritchey @bikebilly

Beside the automated workflow suggested here, could someone do this on a more traditional/manual way as fallback ?

In my case I really need this feature for one domain and its sub-domains. In fact, the project I host on gitlab.com was working perfect, but is now really hurt by this policy with ownership validations during 1 hour sometimes + I need to manually manage my TXT records in order not to reach limits of my DNS provider: quite a regression whilst I was already proven owner of the domain...

Could support help with this issue on a per-case basis or is it totally blocked until this feature happens to be priority ? I doubt a lot of projects actually need this, and I am afraid the feature will just get postponed until... who knows.

I have only one domain I'd like support to unlock all sub-domains for me: can do ? what steps are necessary ?

Thanks :)

@Jonarod we don't have a concept of hierarchy anywhere for Pages domains at present, so there's no one-time hack or fix we could apply manually. Hopefully this can be scheduled quickly though, or at least given ~"Accepting Merge Requests" once we make sure the security dimension is OK.

I see. Thanks for the answer.

Another solution I am thinking of could be to use sub-folders instead of sub-domains to host other sites, but that will have a huge impact for the user base...

Unfortunately I can't wait next release for that. I will probably have to switch back to github.com for this specific project then.

mentioned in issue #43542 (moved)

@nick.thomas I have been trying to figure out some solutions about this and if gitlab doesn't have a concept of hierarchy to hold domain/subdomain relationship, wouldn't it be feasible to allow wildcard subdomain as a domain entry ?

Overview

1. in the UI/API, allow user to add wildcard subdomain: *.mydomain.com. This would then be stored as any other domain entry.
2. Like other entries, a validation code is returned to be placed as a TXT record in the DNS
3. Verification of ownership happens
4. From now on, *.mydomain.com is an accepted pages domain entry for the project.
5. when end-user asks whatever.mydomain.com the routing mechanics would eventually point to the project.

Pros

1. The logic fits with current validation workflow
2. wildcard subdomains are not infered by default, rather it is project owner's decision to reference it as a clear will to route any sub-domains to the same project.
3. No domain/subdomain hierarchy to add

Cons

The routing handler that maps authorized.domain.com to project.gitlab.io has to be modified to either:

• implement a slower regex pattern to match * sub-domain
• have a special isWildcard := true param that will allow any incoming requests to route to project.gitlab.io

Another thing to be careful is to check if *.mydomain.com is already active for any other projects.

Implementation details

I was not able to identify where the domain validation happens in the code for me to suggest something: I'd be glad if one could point me to the handlers for both "new domain entry" and more importantly "domain routing from incoming requests".

@Jonarod I quite like that idea, especially as it dovetails neatly with the lets-encrypt introduction of wildcard certificates: https://community.letsencrypt.org/t/acme-v2-and-wildcard-certificate-support-is-live/55579

The way I'd do this is to add a wildcard boolean to the pages_domains table. So if you have domain: example.com, wildcard: true, then Pages will serve both example.com and www.example.com with the same certificate. Importantly for your use case, www.example.com will be implicitly verified by records added at example.com

If you want TLS on both domains and don't have a wildcard certificate, then you still need to add each domain separately, but I feel this is a nice incremental step, and it doesn't need us to break out verification status and add hierarchical information to it in order to function.

WDYT @bikebilly @ayufan?

There is a downside in that we increase the amount of processing required in the gitlab-pages daemon - if it gets foo.bar.example.com, it has to check for foo.bar.example.com, bar.example.com, example.com and .com before returning a 404 error, but it's not a large cost, since it's all in-memory comparisons.

The inclusion of .com in that list above looks scary, but it isn't unless someone manages to verify the com domain somehow. And a person who could do that would be able to serve arbitrary content on any domain under .com anyway.

In support of the above, it is a bit crazy that we currently ask people to add example.com and www.example.com separately. Perhaps we can even halve the number of custom domains we have this way ;)

We could possible also have the idea of a restricted wildcard, which includes a limited number of arbitrary subdomains, for those who only want example.com and www.example.com. But that might not be necessary.

mentioned in issue #48848 (closed)

added devopsrelease [DEPRECATED] label

added 1 deleted label and removed 1 deleted label

changed milestone to %Backlog

added [deprecated] Accepting merge requests label

Been at this for a couple of days. And I've been among others the following article on your help site, I'm expected to have my site working with both example.com and www.example.com by adding the first as a A(NAME) record, and the latter as a CNAME to the nampespace.gitlab.io.

This is in my zone file:

mindtooth.no.		3600	IN	A	35.185.44.232
www.mindtooth.no.	3600	IN	CNAME	MindTooth.gitlab.io.
_gitlab-pages-verification-code.mindtooth.no TXT ###SHARED CODE###
_gitlab-pages-verification-code.www.mindtooth.no TXT ###SHARED CODE###

Am I doing something wrong, is the documentation wrong, or am I excepting something magical to occur?

(Sorry for posting in this ticket, but I feel it's related to the overall discussion as well.)

added 1 deleted label

Hey!

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks. As part of this migration, issues will be moved to the current gitlab-ee project.

If you have any questions about all of this, please ask them in our dedicated FAQ issue.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

• https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
• https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

Frequently Asked Questions

For an up to date list of questions and answers, please take a look at https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

What will we do with the repository names?

https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.

Why rename gitlab-ce to gitlab-foss?

Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear

I created a merge requests for CE, and this got closed. What do I need to do?

You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.

How does the licensing work in this new setup?

Everything in the ee/ directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in the ee/ directory, the process of contributing changes is the same as when using the gitlab-ce repository.

Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?

No. Merge requests submitted to this project will be closed automatically.

Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?

Yes.

How will this affect users of GitLab CE using Omnibus?

No changes will be necessary, as the packages built remain the same.

How will this affect users of GitLab CE that build from source?

Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.

Where can I see a timeline of the remaining steps?

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304

Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?

Yes, these changes will be synced to gitlab-foss automatically, several times per day.

locked this issue