Wishlist: test anbox/anbox-image/postmarketos-anbox using CI
Quality insurance is hard, and when a piece of software involves a full stack from kernel to userspace with a second operating system in between, that piece of software will break.
I've seen breakages all along the stack: missing kernel modules, cgroups issues, init script failure, LXC failing to mount rootfs, missing dbus, anbox's init script calling exit 1
, android's init script (called by Anbox) erroring out, android's Zygote segfaulting, Anbox's modifications to Android failing to communicate with Anbox and other errors that you can't identify because everything seems to die at the same time.
One day, adb shell su
was working, now it doesn't anymore. And I don't know why.
One day, Anbox was working out of the box on x86_64; then it suddenly couldn't take touch input, so it was unusable. And I don't know why.
One day, Anbox was working out of the box on x86_64. Now it looks like it works but when you try to start an app, the app just bootloops with permission errors everywhere. And I don't know why.
While it is desirable to check the kernel config in pmbootstrap kconfig check [--anbox?]
, the other things can't be checked in such a way. I don't think it is possible to have Anbox working well in pmOS without CI help. At least it would be possible to know which commit broke something. Tests are good anyway. Let's create tests.
Now, to test Anbox, you need the whole stack to be functional and adapted to Anbox, so I don't think a test() in the APKBUILD is suited for that. Fortunately, once the kernel is configured, the packages set up what they need: fuse, cgroups, sysfs, work directory. It would be possible to test it without too much code.
It's a wider issue though: how can we test the distribution in real conditions? Even if we could test each packages (we can't do that for Anbox), I'm sure issues would arise when combining them. Having a way to automate distribution-wide tests with pmOS would be great. Are there initiatives to do that?