Add tests for plugins without tests

We should test the plugins in this repository for what I hope are obvious reasons.

Plugins missing (integration) tests

Elements

• collect_integration - !35 (merged)
• collect_manifest
• distutils - !38 (merged)
• fastboot_bootimg
• fastboot_ext4
• flatpak_image
• flatpak_repo
• makemaker - !40 (merged)
• meson - !36 (merged)
• modulebuild - !41 (merged)
• oci
• qmake
• x86image

Sources

• git_tag
• cargo

Sources missing standardised source tests

• git_tag - !34 (merged)
• cargo

Other things to do

• Add any tests that were missed in the migration of plugins from bst-core - !34 (merged)
• Cleanup the test suite a bit - !34 (merged)

mentioned in merge request !7 (merged)

Any chance we could turn this issue into a check list, might be a bit easier to help us track the issue.

Exactly what I was about to ask ! :) Please could we list all the areas we need to improve?

We can link each point in the list to an MR, and could use this as a flagship issue to track the overall progress.

changed the description

The main difficulty I think we'll have in testing elements is that we need to add dependencies into the project we build here, which means it will almost inevitably get big. For example, I can't think of a way to test the x86image plugin without a dependency on a kernel, or the flatpak_repo plugin without flatpak. As a result, we may need the testing project to become far larger than it currently is, and thus we'll need more cache space for the builds. For examples of the ballooning project size, see bst-external, which builds a minimal x86image using freedesktop-sdk's bootstrap, and also bst-external!84 which adds a test for the flatpak_* plugins.

Perhaps we could add a small cache quota, but then running the tests will take a lot longer as we'll need to rebuild things every time.

mentioned in merge request !35 (merged)

changed the description

marked the checklist item collect_integration - !35 (merged) as completed

marked the checklist item git_tag - !34 (merged) as completed

marked the checklist item Add any tests that were missed in the migration of plugins from bst-core - !34 (merged) as completed

marked the checklist item Cleanup the test suite a bit - !34 (merged) as completed

mentioned in merge request !36 (merged)

mentioned in merge request !38 (merged)

changed the description

mentioned in merge request !40 (merged)

changed the description

mentioned in merge request !41 (merged)

changed the description

Writing tests for the qmake plugin might be a bit difficult, it seems to have a hard dependency on gcc/clang and glibc, however the alpine image uses tcc and musl. As a result, to get qmake to work I'd need to build gcc and glibc, which would likely make the test suite take over an hour longer. This is obviously not ideal.

Maybe we could create a minimal test image with just the things we need with BuildStream?

marked the checklist item distutils - !38 (merged) as completed

marked the checklist item makemaker - !40 (merged) as completed

marked the checklist item meson - !36 (merged) as completed

marked the checklist item modulebuild - !41 (merged) as completed

afaict the following tests are also missing:

• elements:
  • bazel-build
• sources:
  • quilt