Rework bootstrap "sysrooting"

Context

I am not very happy of the I have designed the bootstrap of Freedesktop SDK. One of the reason is that BuildStream does not provide proper support for dependencies in sysroots.

While making an experimental feature implementation on BuildStream, and after discussion with @juergbi on how he does bootstrapping, I have ended finding what Freedesktop SDK can do without to improve bootstrapping without requiring a new feature. However there is still a lot of boiler-plate, but the approach is still much cleaner and fixes several issues.

This work will help for requesting proper feature in BuildStream.

Problems with the current approach

For bootstrapping we need to have a "sysroot". This is where we build all software we need for a minimal image. Once we are finished building the minimum to have a usable stand-alone system, we just take all the files in this sysroot. It becomes the new root (or /).

The current approach is the build all artifacts into "%{sysroot}". For example coreutils.bst provides /usr/bin/ls. But instead of having that file there, the file will effectively be %{sysroot}/usr/bin/ls.

Integration commands in the wrong place

For example ldconfig is called an integration command in bootstrap-extract.bst. It should not be. It should be defined in glibc.bst. However, this would not work because the paths are wrong, everything is in %{sysroot}

bst shell bash.bst does not work

This is a common issue with new-comers. While it is not very useful to have on bash (it works though), it is weird.

Complicated split rules rebuild

Because we go through some compose and script elements to remove %{sysroot} prefix, then we lose original definitions of split rules. The way we handle it now is to make bootstrap-dev-extract.bst and bootstrap-platform-extract.bst, where all files from the former are tagged devel and the files from the latter are removed from devel. Then we compose it with a stack. This is complex. I am not sure many understand what we are doing it other than me.

Split-rules prefixed with %{sysroot}

We have to repeat split rules and prefix them with %{sysroot}. You can see this in bootstrap/project.conf.

Not being able to mix cross-compiled and not cross-compiled.

This is for example interesting for other compilers, like Rust. Rust has much more dependencies, and moving those dependencies.

However Rust depends on things that we do not need to cross-compile.

Description

Instead of having %{sysroot} prefix in artifacts, we instead stage dependencies in the sysroot when needed.

script is the only plugin that can pull dependencies into directories. This is called "layout". So for each element, we split it into 2.

One element is a script, and just has all the dependencies. With the layout, we configure which ones are in the root and with ones are in the sysroot. Note that we could put only the sysroot dependencies in the script. But there is a limitation to requires us to have at least one dependency in the root.

Then the other element is the original elements but with dependencies changed. Now it "build-depend" on the new extracted script element. We need to add "runtime" dependencies because they cannot be provided by the script.

Acceptance Criteria

• No artifact should contain a sysroot.
• We do not use compose or script to depend on the boostrap. We should use a stack instead.