Skip to content

Crash in `bst fetch` with mirroring configured and project.refs storage

Summary

BuildStream crashes with a BUG message and stack trace when fetching a source that:

  • Uses ref-storage: project.refs
  • Does have a ref listed for the given source in the project.refs
  • Has a mirror configured with at least one redirected alias

Relevant logs and/or screenshots

Failure Summary
  sdk/brotli.bst:
    [00:00:00][d17fe05b][fetch:sdk/brotli.bst                ] BUG     Fetch

        An unhandled exception occured:
    
        Traceback (most recent call last):
          File "/codethink/GNOME/buildstream/buildstream/_scheduler/jobs/job.py", line 407, in _child_action
            result = self.child_process()
          File "/codethink/GNOME/buildstream/buildstream/_scheduler/jobs/elementjob.py", line 94, in child_process
            return self._action_cb(self._element)
          File "/codethink/GNOME/buildstream/buildstream/_scheduler/queues/fetchqueue.py", line 44, in process
            source._fetch()
          File "/codethink/GNOME/buildstream/buildstream/source.py", line 630, in _fetch
            new_source.fetch()
          File "/codethink/GNOME/buildstream/buildstream/plugins/sources/_downloadablefilesource.py", line 80, in fetch
            if os.path.isfile(self._get_mirror_file()):
          File "/codethink/GNOME/buildstream/buildstream/plugins/sources/_downloadablefilesource.py", line 171, in _get_mirror_file
            return os.path.join(self._get_mirror_dir(), sha or self.ref)
          File "/usr/lib/python3.5/posixpath.py", line 89, in join
            genericpath._check_arg_types('join', a, *p)
          File "/usr/lib/python3.5/genericpath.py", line 143, in _check_arg_types
            (funcname, s.__class__.__name__)) from None
        TypeError: join() argument must be str or bytes, not 'NoneType'

Possible fixes

Fix source re-instantiation involved with fetching with mirroring enabled, such that the resulting newly instantiated Source has passed through the same loading codepaths as it does normally.

Other relevant information

  • BuildStream version affected: /milestone %BuildStream_v1.x

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information