Skip to content

Draft: Make the post-processing script faster

Achilleas Pipinellis - OOO until November 25th (parental leave) requested to merge axil-speedy-gonzales into main

What does this MR do and why?

Use multiple procecess with xargs. From the manpage:

  • -n max-args: Use at most max-args arguments per command line.
  • -P max-procs: Run up to max-procs processes at a time. If max-procs is 0, xargs will run as many processes as possible at a time.

Before

After

❯ rm -rf 15.10 && cp -a public 15.10 && time scripts/normalize-links.sh 15.10 15.10
INFO: Replacing relative URLs in 15.10 for HTML files...
INFO: Replacing relative URLs in 15.10 for CSS files...
INFO: Replacing relative URLs in 15.10 for JavaScript files...
INFO: Replacing full URLs in 15.10 for HTML files...
INFO: Fixing URLs inside the sitemap...
scripts/normalize-links.sh 15.10 15.10  84.30s user 11.76s system 670% cpu 14.334 total

How to set up and validate locally

  1. Remove public and rebuild the site:

    make clean && make compile

  2. Copy public to a version in order to compare the before and after:

    cp -a public 15.10

  3. Run the script in main:

    time scripts/normalize-links.sh 15.10 15.10

  4. Check out this branch.

  5. Run the script in this branch:

    rm -rf 15.10 && cp -a public 15.10 && time scripts/normalize-links.sh . 15.9

  6. Optional. You can use diff to check that the results are the "same" (strings replaced, the only difference should be the version).

    diff 15.9 15.10

Merge request acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports