Broken external links are not detected by CI
Description
Broken external link detection (test_external_links in the Test stage) is not detecting all of the broken links. The broken links are detected when running bundle exec nanoc check external_links
locally
Steps to reproduce
- navigate to https://docs.gitlab.com/ee/administration/gitaly/
- search for "tailing the logs"
- Click the link
- receive 404 (the link https://docs.gitlab.com/omnibus/settings/logs.md is not found)
Actual result
https://docs.gitlab.com/omnibus/settings/logs.md is not reported as a 404 link
Expected result
https://docs.gitlab.com/omnibus/settings/logs.md should be reported as a broken link (receives 404)
Further details
I added the commands (for a different broken link on the https://docs.gitlab.com/ee/university/support/#learn-about-the-escalation-process-for-tickets page - search for "Read about Escalation"):
grep issue_escalation public/ee/university/support/README.html
curl -Is https://about.gitlab.com/handbook/support/workflows/shared/support_workflows/issue_escalations.html
to .gitlab.ci.yml and received the following output:
$ grep issue_escalation public/ee/university/support/README.html
<li>Read about <a href="https://about.gitlab.com/handbook/support/workflows/shared/support_workflows/issue_escalations.html">Escalation</a>
$ curl -Is https://about.gitlab.com/handbook/support/workflows/shared/support_workflows/issue_escalations.html
HTTP/2 404
x-guploader-uploadid: AEnB2UpIsnLkRHCoHccVV0Fgi68vvr-_0_zXZoguL8LdGitg6r5ec9yXrzBgg8RDVlTPNTYHshLaguLRJ67letk2oFuvnAwFDA
expires: Wed, 11 Sep 2019 14:22:35 GMT
cache-control: private, max-age=0
last-modified: Sun, 01 Sep 2019 16:04:23 GMT
etag: "3ea8d867f6afee0e45745ef204682ccf"
x-goog-generation: 1567353863348785
x-goog-metageneration: 1
x-goog-stored-content-encoding: identity
x-goog-stored-content-length: 46150
x-goog-meta-goog-reserved-file-mtime: 1567352693
content-type: text/html
x-goog-hash: crc32c=N8A1eA==
x-goog-hash: md5=PqjYZ/av7g5FdF7yBGgszw==
x-goog-storage-class: MULTI_REGIONAL
accept-ranges: bytes
server: UploadServer
accept-ranges: bytes
age: 0
accept-ranges: bytes
date: Wed, 11 Sep 2019 14:22:35 GMT
via: 1.1 varnish
age: 0
x-served-by: cache-dca17748-DCA
x-cache: MISS
x-cache-hits: 0
x-timer: S1568211755.973388,VS0,VE56
vary: Accept-Encoding,Origin
content-security-policy: frame-ancestors 'self' https://gitlab.lookbookhq.com https://learn.gitlab.com;
strict-transport-security: max-age=300
content-length: 46150
So it appears that the link will return 404 if it's checked. I've run out of ideas for finding this issue