Skip to content

WIP: Make pages to use API

This is Proof of Concept, but surprisingly complete Merge Request to make Pages to use API interface with GitLab.

Pages for each domain, either GitLab Domain or Custom Domain do request the configuration from GitLab. The configuration describes certificate to be used, and a list of paths from which to serve data. This retains the backward compatibility for all existing features with almost no changes to tests, and implements a generic storage driver for easy addition for additional storage providers that are not file-system based.

This MR effectively simplifies GitLab Pages daemon, as it removes a lot of logic to query a disk-based database, to construct lookup paths, and find files based on a number of criteria.

This MR makes Pages to be dumb in terms what type of domain it serves, as it removes the distinction between custom and group pages. The handling of domains is being controlled by GitLab API, providing an ordered list of paths to evaluate. The evaluation works on first-find-first-served. You can see an example specification by looking at internal/client/mock_api.go. This Mock is being used by tests, and represents the configuration stored in directories and in config.json.

The API to be used (currently mocked) is:

>> /api/v4/pages/domain
>>> ?host=my-group.gitlab-example.io
<<< {"lookup_paths":[{"prefix":"/CapitalProject/", "path":"group/CapitalProject/public/"}]}

The current implementation makes it compatible with disk-based storage, it does not try to implement the S3-based storage, as this is now possible with pluggable interface present in ./internal/storage.

What is missing for this Merge Request to not become PoC?

  1. Implement GitLab Rails API,
  2. Implement caching of GitLab Rails API requests, the idea here to cache the responses for 60s,
  3. Cover with tests the ./internal/client,
  4. Provide a way to authorize GitLab Pages requests between Pages Daemon and Rails API, currently, it is unauthenticated.

This was done as part of research to understand the complexity of changing Pages architecture to solve:

  1. The slow start of Pages,
  2. Prepare for scalable Pages architecture, Pages living on external storage.
Edited by Kamil Trzciński

Merge request reports

Loading