Optimize caching for protected pages

Summary

Pages is cached by time rather than change. This causes inconsistencies when browsing pages before and after making changes.

Steps to reproduce

  1. Set up a project using pages in protected mode
  2. Visit the page
  3. Make some changes to the page
  4. Visit the page again (within 10 minutes of step 2). The page is still at the old version.

What is the current bug behavior?

In HTTP responses the cache-control header is set to max-age=600. There's two problems with this (IMO):

  1. Changes does not propagate to users within 600 seconds since last visit.
  2. Cached pages that did not change still expires after 600 seconds.

What is the expected correct behavior?

Pages should use cache validation. Preferably using the Cache-Control: must-revalidate header. Alternatively the simpler Last-Modified header.

Output of checks

This bug happens on GitLab.com / gitlab.io

Possible fixes

https://gitlab.com/gitlab-org/gitlab-pages/-/blob/master/internal/serving/disk/reader.go#L221