Skip to content

Workspace builds do not correctly determine that they are not cached

Summary

If you delete the cache file eg. ~/.cache/buildstream/artifacts/cas/refs/heads/outOfSourceBuilds2/hello/06237fbe18a8674f5e938129cc8a8eafafa951b48082e59543e5650466e5fd01 for a non workspace build then it will trigger a rebuild, (this is one of the few ways to trigger a rebuild of a cached build for testing.)

However if the element has a workspace it is unable to detect the missing cache file and trys to open it resulting in a file not found error

This bug can also be triggered by trying to build a failing element (with a workspace) twice.

Steps to reproduce

steps:

  • Create a workspace for a element you can build
  • Build the element
  • Rebuild the element, nothing will build but it will show you the key it has found
  • Go to ~/.cache/buildstream/artifacts/cas/refs/heads/projectName/elementName
  • There should be a file that has the name of the cache key, delete it.
  • Go back to your project and rebuild the element
  • You should get a bug trace with:
self, self._get_brief_display_key())
    AssertionError: autotools element at hello.bst [line 1 column 0]: Missing artifact ????????

What is the current bug behaviour?

bug message

What is the expected correct behaviour?

rebuild triggered, like for non workspace builds.

Relevant logs and/or screenshots

 will  CT-will-S-Deb  ~  projects  …  doc  examples  out-of-source-build-helloworld 
 willsalmon/simpleOutSource  3✎  9?  $  bst build hello.bst
[--:--:--][][] START   Build
[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] BUG     autotools element at hello.bst [line 1 column 0]: Missing artifact ????????

    Traceback (most recent call last):
      File "/home/will/.local/bin/bst", line 8, in <module>
        sys.exit(cli())
      File "/home/will/.local/lib/python3.5/site-packages/click/core.py", line 722, in __call__
        return self.main(*args, **kwargs)
      File "/home/will/projects/buildstream/buildstream/buildstream/_frontend/cli.py", line 162, in override_main
        standalone_mode=standalone_mode, **extra)
      File "/home/will/.local/lib/python3.5/site-packages/click/core.py", line 697, in main
        rv = self.invoke(ctx)
      File "/home/will/.local/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/will/.local/lib/python3.5/site-packages/click/core.py", line 895, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/will/.local/lib/python3.5/site-packages/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "/home/will/.local/lib/python3.5/site-packages/click/decorators.py", line 27, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/home/will/projects/buildstream/buildstream/buildstream/_frontend/cli.py", line 330, in build
        build_all=all_)
      File "/home/will/projects/buildstream/buildstream/buildstream/_stream.py", line 184, in build
        dynamic_plan=True)
      File "/home/will/projects/buildstream/buildstream/buildstream/_stream.py", line 933, in _load
        selected = self._pipeline.get_selection(elements, selection, silent=False)
      File "/home/will/projects/buildstream/buildstream/buildstream/_pipeline.py", line 209, in get_selection
        elements = self.plan(targets)
      File "/home/will/projects/buildstream/buildstream/buildstream/_pipeline.py", line 178, in plan
        return _Planner().plan(elements, plan_cached)
      File "/home/will/projects/buildstream/buildstream/buildstream/_pipeline.py", line 485, in plan
        self.plan_element(root, 0)
      File "/home/will/projects/buildstream/buildstream/buildstream/_pipeline.py", line 476, in plan_element
        if not element._cached_success():
      File "/home/will/projects/buildstream/buildstream/buildstream/element.py", line 973, in _cached_success
        return self.__cached_success(keystrength=None)
      File "/home/will/projects/buildstream/buildstream/buildstream/element.py", line 2643, in __cached_success
        success, _, _ = self.__get_build_result(keystrength=keystrength)
      File "/home/will/projects/buildstream/buildstream/buildstream/element.py", line 2635, in __get_build_result
        self.__load_build_result(keystrength)
      File "/home/will/projects/buildstream/buildstream/buildstream/element.py", line 2615, in __load_build_result
        self.__assert_cached(keystrength=keystrength)
      File "/home/will/projects/buildstream/buildstream/buildstream/element.py", line 2092, in __assert_cached
        self, self._get_brief_display_key())
    AssertionError: autotools element at hello.bst [line 1 column 0]: Missing artifact ????????

Possible fixes

the element seems to think it is cached, this then triggers the fetch. I don't know were the non workspace path differers but it looks like it is in the determining if cached state as the elements are reporting that they are cached, I have not checked that non workspace element know they are not cached at that point.

Other relevant information

This was discovered while working on #512 (closed) but fixing this is not required to close it.

  • BuildStream version affected: master (c6155f8d) /milestone %BuildStream_v1.x

Edited by Tristan Van Berkom
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information