Buildstream tries to use message without a message handler if linux platform and no user namespace support

Summary

If buildstream is using the linux platform but user namespaces aren't supported, it will try to use the message handler before it's set, and fail an assertion.

Steps to reproduce

  1. Use a linux system that doesn't use user namespaces.
  2. Try to bst fetch anything.

I replicated this by running the benchmark in https://gitlab.com/BuildStream/benchmarks (i.e. python3 -m bst_benchmarks)

What is the current bug behavior?

In App.initialized() when bst fetch is called, it tries to instantiate the Platform (https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_frontend/app.py#L201), and sets the message handler on line 215. If the platform doesn't support user namespaces, it will try to use the message handler to print a message before it's set a message handler.

What is the expected correct behavior?

It does not fail an assertion that there is a message handler set before it tries to send a message.

Relevant logs and/or screenshots

Console output

jonathanmaw@fafnir:~/workspace/buildstream/benchmarks$ python3 -m bst_benchmarks -c configs/simple.benchmark
INFO:root:BuildStream benchmark runner started at 2018-09-20 17:31:36.593847
INFO:root:Preparing image 'bst_benchmarks:master'
^C
Aborted!
jonathanmaw@fafnir:~/workspace/buildstream/benchmarks$ python3 -m bst_benchmarks
INFO:root:BuildStream benchmark runner started at 2018-09-20 17:31:43.612275
INFO:root:Preparing image 'bst_benchmarks:1.0.1'
INFO:root:Preparing image 'bst_benchmarks:1.1.0'
INFO:root:Preparing image 'bst_benchmarks:stable'
INFO:root:Preparing image 'bst_benchmarks:master'
INFO:root:Preparing volume 'baserock-source'
Volume setup script exited with code 1. Error output:
+ git clone https://gitlab.com/BuildStream/bst-external/
Cloning into 'bst-external'...
warning: redirecting to https://gitlab.com/BuildStream/bst-external.git/
+ cd bst-external
+ git checkout c4cc10467d116450471ca9f756617ede7572814c
Note: checking out 'c4cc10467d116450471ca9f756617ede7572814c'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at c4cc104 Merge branch 'sam/docker-source' into 'master'
+ pip3 install --user .
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Processing /bst-external
Collecting requests (from BuildStream-external==0.1)
  Downloading https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl (91kB)
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (from BuildStream-external==0.1)
Collecting urllib3<1.24,>=1.21.1 (from requests->BuildStream-external==0.1)
  Downloading https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl (133kB)
Collecting certifi>=2017.4.17 (from requests->BuildStream-external==0.1)
  Downloading https://files.pythonhosted.org/packages/df/f7/04fee6ac349e915b82171f8e23cee63644d83663b34c539f7a09aed18f9e/certifi-2018.8.24-py2.py3-none-any.whl (147kB)
Collecting chardet<3.1.0,>=3.0.2 (from requests->BuildStream-external==0.1)
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting idna<2.8,>=2.5 (from requests->BuildStream-external==0.1)
  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)
Installing collected packages: urllib3, certifi, chardet, idna, requests, BuildStream-external
  Running setup.py install for BuildStream-external: started
    Running setup.py install for BuildStream-external: finished with status 'done'
Successfully installed BuildStream-external-0.1 certifi-2018.8.24 chardet-3.0.4 idna-2.7 requests-2.19.1 urllib3-1.23
+ cd ..
+ mkdir -p /root/.config
+ echo 'sourcedir: /src'
+ git clone https://gitlab.com/baserock/definitions
Cloning into 'definitions'...
warning: redirecting to https://gitlab.com/baserock/definitions.git/
+ cd definitions
+ git checkout 9da3339c8b3d5080fcacc2f3f9a38178e3c9879c
fatal: reference is not a tree: 9da3339c8b3d5080fcacc2f3f9a38178e3c9879c
+ bst fetch gnu-toolchain/stage1-binutils.bst
bwrap: No permissions to creating new namespace, likely because the kernel does not allow non-privileged user namespaces. On e.g. debian this can be enabled with 'sysctl kernel.unprivileged_userns_clone=1'.
Traceback (most recent call last):
  File "/usr/local/bin/bst", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib64/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 162, in override_main
    standalone_mode=standalone_mode, **extra)
  File "/usr/local/lib64/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib64/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib64/python3.6/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_frontend/cli.py", line 376, in fetch
    with app.initialized(session_name="Fetch"):
  File "/usr/lib64/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_frontend/app.py", line 201, in initialized
    Platform.create_instance(self.context)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_platform/platform.py", line 61, in create_instance
    cls._instance = PlatformImpl(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_platform/linux.py", line 38, in __init__
    self._user_ns_available = self._check_user_ns_available(context)
  File "/usr/local/lib/python3.6/site-packages/buildstream/_platform/linux.py", line 82, in _check_user_ns_available
    "artifacts created will not be trusted for push purposes."))
  File "/usr/local/lib/python3.6/site-packages/buildstream/_context.py", line 354, in message
    assert self._message_handler
AssertionError

Possible fixes

  • Move the set_message_handler call to before creating the platform.

or

  • Use something other than context.message() to warn that user namespaces don't work.

Other relevant information

Edited by Jonathan Maw