Render warmer: roadmap to production

The Warmer module-based preemptive caching system we have for wiki node changes works really well, which can be in sharp contrast to the multiple seconds it can take to render some of the larger non-changes pages; adapting the the changes pre-rendering to pre-cache all the wiki nodes for all the available permission hashes and other cache contexts would help page load performance immensely.

The Warmer View Mode module at first seemed like a good fit for this, but it doesn’t support multiple cache contexts, nor does it offer a hook or event for us to provide this information to it. An issue was opened and the module maintainer weighed in on this limitation; in the short term, it seems like prototyping/refining this in our own custom module is the best way to go, and in the long term, we can contribute a more general purpose implementation back to the module if possible.

Requirements

  1. We need to be able to render not just for anonymous users, but for all available combinations of permission hashes, i.e. authenticated users with different roles.
  2. Because of 1., we likely can't use Guzzle to make requests to warm caches that way, since that would only work for anonymous users unless we resorted to some really hacky session cookie stuff that would be a security nightmare. This should be able to run internally without having to make external requests to the site.

Implementation details and challenges

The changes rendering gets cached in our own cache bin rather than Drupal’s render cache, so we have a bit more control over it, making it easier to get a cache hit since we have our own controller and services to handle this. When using Drupal core’s render cache, we may end up with cached items not matching requests in unexpected ways and so cache misses/wasted warming; this would be due to different and unexpected cache contexts. We have two primary ways we could address this:

  1. Implement our own cache for wiki nodes; this would allow control on par with the changes rendering and would be ideal, but it requires us to re-implement a basic form of what the render cache does with contexts and so on.
  2. Use Drupal core’s render cache and test thoroughly/continually that the warmed cache items result in cache hits in expected user permission hashes; this is probably not as great a solution because it may become a whack-a-mole game to try and keep the expected cache contexts if core changes something.

Unexpected blockers

The current work-in-progress implementation has run into the following blockers:

  1. While the rendering occurs successfully, they don’t result in a cache hit because of additional cache contexts that are missing:
    1. timezone: After a lot of searching, this seems to be added in \Drupal\datetime\Plugin\Field\FieldFormatter\DateTimeFormatterBase::buildDate() - this might be due to us having date fields on wiki nodes, but those are set as disabled on all view modes so they should not be rendering. The ideal solution here would be to remove this cache context since we don’t vary wiki nodes by real world time zones.
      1. See also \Drupal\Core\Cache\Context\TimeZoneCacheContext
    2. route: This is pretty self explanatory. The route cache context (\Drupal\Core\Cache\Context\RouteCacheContext) varies by the route name and parameters, which is a problem when rendering without the expected route. We have several potential solutions:
      1. Remove this cache context so it’s no longer applied. This is probably a bad idea and may have unintended consequences.
      2. Figure out how to set the route and route parameters before rendering the render array. This might also be a bad idea and could cause unexpected issues.
      3. Rewrite/alter the cache metadata before rendering. Replicating what the route cache context class does (hash the parameters, etc.) would be an easy copy and paste, though not ideal. Actually applying this to the cache ID might be more challenging and require a low level understanding of the render cache.

Adding

$renderArray['#cache']['contexts'][] = 'timezone';
$renderArray['#cache']['contexts'][] = 'route';

causes the cache IDs to have the respective contexts added as cache keys as expected, but of course this only generates values based on the current render, which would be the current user's time zone and no route (maybe front page/root path?) when run via Drush.

See also

While not required for this, it may be a good time to implement neurocracy/drupal-omnipedia-user#2 as rendering as existing user accounts might only exacerbate potential security issues when using this technique for all wiki node rendering.

Drupal core

It looks like the contrib VariationCache module, which basically provides a cache context framework separate from the render cache - will be in Drupal core 10.2.

Third-party links

Edited by Ambient.Impact