Skip to content

Docs: enable parallel processing

Closed, but check the follow-up MR !4368 (merged).

Context

This makes sphinx-build execute in parallel.

All that's needed is to pass the -j auto. Furthermore I had to declare our custom roles as safe for parallel reading: I'm not sure exactly what that entails (no documentation) but since each of the roles have a local effect (per-file) I think it should be fine.

So what's the speed-up? Not... incredible. On my 8 core machine:

Sequential:

$ for i in $(seq 1 5); do rm -rf _build; /usr/bin/time poetry run sphinx-build -b html . "_build" 2>&1  | grep elapsed; done        
58.93user 23.15system 1:22.97elapsed 98%CPU (0avgtext+0avgdata 123680maxresident)k
59.92user 22.96system 1:23.76elapsed 98%CPU (0avgtext+0avgdata 127980maxresident)k
59.68user 23.67system 1:24.24elapsed 98%CPU (0avgtext+0avgdata 127968maxresident)k
60.09user 23.05system 1:24.04elapsed 98%CPU (0avgtext+0avgdata 127788maxresident)k
59.11user 22.96system 1:22.96elapsed 98%CPU (0avgtext+0avgdata 125496maxresident)k

In parallel (p=8):

$ for i in $(seq 1 5); do rm -rf _build; /usr/bin/time poetry run sphinx-build -j auto -b html . "_build" 2>&1  | grep elapsed; done
69.28user 24.84system 1:06.20elapsed 142%CPU (0avgtext+0avgdata 147092maxresident)k
69.29user 24.24system 1:06.41elapsed 140%CPU (0avgtext+0avgdata 147244maxresident)k
66.49user 24.48system 1:04.77elapsed 140%CPU (0avgtext+0avgdata 146528maxresident)k
73.49user 24.67system 1:07.90elapsed 144%CPU (0avgtext+0avgdata 146944maxresident)k
66.71user 24.12system 1:05.49elapsed 138%CPU (0avgtext+0avgdata 146416maxresident)k

Manually testing the MR

Check out the docs by the CI or run make -C docs and read the results.

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Nic Volanschi

Merge request reports