Native bst support on Windows (not WSL)

Windows is a primary development platform for me, I want to be able to use BuildStream on Windows to build binaries for Windows.

Previously, we completed this ticket for remote-only support on WSL: #412 (closed)

I aired this idea on the mailing list here, with a TODO list of primary concerns.

Initial POC

Can compile using host tools, using a 'zero sandbox'.

• !1298 (closed): Proof-of-concept of BuildStream building things on Windows.

Rebased POC, after introduction of buildbox-casd, only supporting import elements

Only supports import elements, tested on doc/examples/first-project/.

• Experimental branches for support buildbox-casd on Windows, working for 'import-only' workflow: BuildGrid/buildbox/buildbox-common!134, BuildGrid/buildbox/buildbox-casd!89 (closed).
• Make a new minimal example of BuildStream working on Windows: https://gitlab.com/BuildStream/buildstream/commits/aevri/win32_minimal
• #1154 (closed): Independently confirm the Windows experiment works.

Official win32 support, using a 'zero sandbox'

Land the changes necessary to be able to run doc/examples/first-project/ on Windows.

• #1163 (closed), !1678 (merged): Add CI for native windows support.
• Full support for multiprocessing.set_start_method('spawn'), for platforms that don't support fork.
  • !1663 (closed): Enable pytest tests/ with BST_FORCE_START_METHOD=spawn
  • !1675 (closed): Enable all tests for BST_FORCE_START_METHOD=spawn
  • Measure performance of spawn mode, determine if we should be mindful of what references what, so that we're not pickling more than we need to. This may mean being more explicit in general about the split between the main and worker processes.
• Change to use some more portable versions of things, re-assess some assumptions, being mindful of performance across platforms. Including:
  • Separators: consider os.sep and os.altsep
  • Signals, SIGINT, SIGTERM, SIGPIPE not on win32.
  • Use of add_child_handler to wait on subprocesses, not available on win32.
  • Use of loop.add_reader(queue._reader.fileno() ...) (maybe this works? add_reader and _reader seem to be available on Win32).
  • Takes a couple of seconds to spawn a BuildStream process on Windows.
  • Can't use UNIX domain sockets to communicate with buildbox-casd on Windows, grpc doesn't have that support, yet.
  • ...
  • See !1633 (closed): WIP: win32: support 'bst show' with buildbox-casd and just enough signal support
• Support CASCache.import_directory in buildbox-casd on Windows.
• Fully support buildbox-casd on Windows.
• Introduce SandboxNone, which relies on copying files and good faith. I'd also like to try this on MacOS, to provide an alternative to the package manager Homebrew.
• Bonus: Provide a facility for plugin developers to record and replay job invocations, for debugging. This works by saving the pickled job data and launching it again separately.

Proper sandboxing on Windows

• Evaluate ProjFS on Windows, to provide some filesystem sandboxing.
• Evaluate Windows Sandbox for sandboxing.
• Evaluate Windows Containers for sandboxing.