Skip to content

Make gitlab client cache configurable

The internal/source/gitlab/cache/ package is currently semi hardcoded using the defaultCacheConfiguration. This makes it hard to write acceptance tests that can execute faster. For example !393 (comment 463829311).

We should introduce extra flags that will allow configuring the cache's behaviour for resolving a domain's configuration.

Default configuration

var defaultCacheConfig = cacheConfig{
	cacheExpiry:          10 * time.Minute,
	entryRefreshTimeout:  60 * time.Second,
	retrievalTimeout:     30 * time.Second,
	maxRetrievalInterval: time.Second,
	maxRetrievalRetries:  3,
}

Implementation

  1. Add the following flags to Pages and override defaultCacheConfig
  • gitlab-cache-expiry -> The maximum time a domain's configuration is stored in the cache (default: 600s)
  • gitlab-cache-refresh -> The interval at which a domain's configuration is set to be due to refresh (default: 60s)
  • gitlab-cache-cleanup -> The interval at which expired items are removed from the cache (default: 60s)
  • gitlab-retrieval-timeout -> The maximum time to wait for a response from the GitLab API per request (default: 30s)
  • gitlab-retrieval-interval -> The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: 1s)
  • gitlab-retrieval-retries -> The maximum number of times to retry to resolve a domain's configuration via the API (default: 3)
  1. Move cacheConfig to internal/config/
  2. Add flags to Omnibus
  3. Documentation updates

The first part of #507 (closed) is done so we can work on this

What do these settings do

Users can configure these settings when using API-based configuration to modify the cache behavior for a domain's resolution. The recommended default values are set inside GitLab Pages and should only be modified if needed.

Some examples:

  • Increasing gitlab-cache-expiry will allow items to exist in the cache longer. This setting might be useful if the communication between Pages and GitLab Rails is not stable or the content served by Pages does not change frequently.
  • Increasing gitlab-cache-refresh will reduce the frequency at which GitLab Pages requests a domain's configuration from GitLab Rails. This setting might be useful for content that does not change frequently.
  • Decreasing gitlab-retrieval-retries will report issues between Pages and Rails more quickly. However, they can be transient failures rather than real issues.
Edited by Jaime Martinez