Support internationalised domain names in Pages custom domains

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Per https://gitlab.com/gitlab-com/support-forum/issues/3091

IDNs are those domains that contain characters outside of the ASCII character set, e.g. 肉.com -> xn--r00a.com

Currently, custom domains (including verification) should work if you enter the latter representation explicitly - although we should check that explicitly and add specs for it. The gitlab-pages daemon may need to check internationalized Host: headers and SNI indicators, converting them to the punycode representation before lookup, but I think it's standards-compliant for browsers to send the punycode representation in both cases.

Verification, at the very least, doesn't work for IDNs. I suspect that Pages generally is broken for them too. We should be able to fix that by transparently converting IDNs to their punycode representation before adding them to the database.

Punycoded domains can be displayed as their IDN equivalent, to improve the user experience, but certainly anything that goes to DNS or the filesystem should be punycoded.

We also need to ensure that our TLS setup handles the IDN case. This may require changes in the pages daemon also.

Edited by 🤖 GitLab Bot 🤖