Skip to content

Publish deployed artifacts from master build

Chad Woolley requested to merge deploy-latest-master-to-review into master

What does this MR do?

Upload artifacts from latest master build as a baseline to "prime" the bucket when deploying MR review apps.

Why is this important?

  1. It will cut review app deploy times almost in half. Comparison:
  • The old way with a "non-primed" review app initial upload uploads 12.7k files (1.4 GiB) and takes almost 9 minutes
  • This MR with a "primed" review app initial upload uploads 2.6k files (207.0 MiB) and takes just over 5 minutes
  • If we include the Disqus comments in the review apps, this takes it even further down to 239 files (7.7 MiB), and just over 4 minutes. At this point, the actual deploy is only taking 51 seconds.
  1. This facilitates an improved replacement for the fast "partial build and deploy" approach that is already used for the specially-named "blog post" MR approach. It is an improvement because:
    • The entire review site will be usable, not just the part that got built and deployed.
    • The custom special-MR-name approach can be replaced with a rules-based changes approach, which will be simpler and can be applied to all jobs and sub-sites in a standardized way (the actual implementation of that will be a separate MR)

Tasks

  • Implement the job, test it, see that it significantly improves deploy time of review app.
  • Make the review job block on it via needs. This is not straightforward, because the review job is already at the current max limit of 10 needs entries. **Fixed by switching to dependencies instead of needs for now.

Remaining questions

  • Why did it still need to upload 2.6k files??? There are no content changes in this MR, and it was up-to-date with master 🤔
  • Do we need to wait for this bug to get fixed? I don't think so, since this job is set to allow failure, if it fails, the review will still work but just take up to the previous normal length of time.
  • How concerned are we about errors due to incompatibilities if people are very out of date with master? I don't think very much, Middleman should handle rebuilding any dependencies that are needed, which will be overwritten over the old master versions. Assets (javascripts and stylesheets) might be the most likely to cause problems, but those problems will be addressed in separate MR(s).

Related issues

Relates: !49086 (closed)

Edited by Chad Woolley

Merge request reports