Unhandled exception freezing the build

Summary

We encountered an error in the Freedesktop Sdk CI.

After the traceback, the build got stuck in that state for a long time, until we manually killed it.

Steps to reproduce

This is the first time it ever happens to us, we have no idea how to reproduce it.

What is the current bug behavior?

Invalid text printed to stdout, a Python exception leading to a second Python exception, and a build stuck seemingly forever.

What is the expected correct behavior?

The build should have completed. (and given how far along it was, it most likely would have succeeded)

Relevant logs and/or screenshots

$ bst --colors -o target_arch x86_64 build all.bst
[… snip …]
x����H������H���P�^��UD����[00:00:37][65fa531c][build:desktop-platform-image.bst    ] BUG     Build

    An unhandled exception occured:
    
    Traceback (most recent call last):
      File "/root/.local/lib/python3.6/site-packages/buildstream/_fuse/mount.py", line 148, in mounted
        yield
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_sandboxbwrap.py", line 212, in run
        self.try_remove_device(device_path)
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_sandboxbwrap.py", line 390, in try_remove_device
        os.unlink(device_path)
    OSError: [Errno 107] Transport endpoint is not connected: '/builds/freedesktop-sdk/freedesktop-sdk/cache/buildstream/build/desktop-platform-image-v278qds4/scratch/_/mount/dev/full'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/root/.local/lib/python3.6/site-packages/buildstream/_scheduler/jobs/job.py", line 415, in _child_action
        result = self.child_process()
      File "/root/.local/lib/python3.6/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 94, in child_process
        return self._action_cb(self._element)
      File "/root/.local/lib/python3.6/site-packages/buildstream/_scheduler/queues/buildqueue.py", line 35, in process
        return element._assemble()
      File "/root/.local/lib/python3.6/site-packages/buildstream/element.py", line 1498, in _assemble
        collect = self.assemble(sandbox)
      File "/root/.local/lib/python3.6/site-packages/buildstream/plugins/elements/compose.py", line 125, in assemble
        dep.integrate(sandbox)
      File "/root/.local/lib/python3.6/site-packages/buildstream/element.py", line 794, in integrate
        exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_sandboxbwrap.py", line 247, in run
        pass
      File "/usr/lib64/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib64/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib64/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/usr/lib64/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib64/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib64/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/root/.local/lib/python3.6/site-packages/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/root/.local/lib/python3.6/site-packages/buildstream/_fuse/mount.py", line 150, in mounted
        self.unmount()
      File "/root/.local/lib/python3.6/site-packages/buildstream/_fuse/mount.py", line 128, in unmount
        .format(type(self).__name__, self.__process.exitcode))
    buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

See the full build log at https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/116255830

Possible fixes

No idea.

• BuildStream version affected: 1.2.3

---