Native bst support on Windows (not WSL)
Windows is a primary development platform for me, I want to be able to use BuildStream on Windows to build binaries for Windows.
Previously, we completed this ticket for remote-only support on WSL: #412 (closed)
I aired this idea on the mailing list here, with a TODO list of primary concerns.
Initial POC
Can compile using host tools, using a 'zero sandbox'.
-
!1298 (closed): Proof-of-concept of BuildStream building things on Windows.
buildbox-casd
, only supporting import elements
Rebased POC, after introduction of Only supports import elements, tested on doc/examples/first-project/
.
-
Experimental branches for support buildbox-casd on Windows, working for 'import-only' workflow: BuildGrid/buildbox/buildbox-common!134, BuildGrid/buildbox/buildbox-casd!89 (closed). -
Make a new minimal example of BuildStream working on Windows: https://gitlab.com/BuildStream/buildstream/commits/aevri/win32_minimal -
#1154 (closed): Independently confirm the Windows experiment works.
Official win32 support, using a 'zero sandbox'
Land the changes necessary to be able to run doc/examples/first-project/
on Windows.
-
#1163 (closed), !1678 (merged): Add CI for native windows support. -
Full support for multiprocessing.set_start_method('spawn')
, for platforms that don't supportfork
.-
!1663 (closed): Enable pytest tests/
withBST_FORCE_START_METHOD=spawn
-
!1675 (closed): Enable all tests for BST_FORCE_START_METHOD=spawn
-
Measure performance of spawn mode, determine if we should be mindful of what references what, so that we're not pickling more than we need to. This may mean being more explicit in general about the split between the main and worker processes.
-
-
Change to use some more portable versions of things, re-assess some assumptions, being mindful of performance across platforms. Including: - Separators: consider
os.sep
andos.altsep
- Signals, SIGINT, SIGTERM, SIGPIPE not on win32.
- Use of add_child_handler to wait on subprocesses, not available on win32.
- Use of
loop.add_reader(queue._reader.fileno() ...)
(maybe this works?add_reader
and_reader
seem to be available on Win32). - Takes a couple of seconds to spawn a BuildStream process on Windows.
- Can't use UNIX domain sockets to communicate with buildbox-casd on Windows, grpc doesn't have that support, yet.
- ...
- See !1633 (closed): WIP: win32: support 'bst show' with buildbox-casd and just enough signal support
- Separators: consider
-
Support CASCache.import_directory
in buildbox-casd on Windows. -
Fully support buildbox-casd on Windows. -
Introduce SandboxNone, which relies on copying files and good faith. I'd also like to try this on MacOS, to provide an alternative to the package manager Homebrew. -
Bonus: Provide a facility for plugin developers to record and replay job invocations, for debugging. This works by saving the pickled job data and launching it again separately.
Proper sandboxing on Windows
- Evaluate ProjFS on Windows, to provide some filesystem sandboxing.
- Evaluate Windows Sandbox for sandboxing.
- Evaluate Windows Containers for sandboxing.
Edited by Jürg Billeter