PermissionError when calling `vdirectory.import_files()`

Overview

Using master, my build instantly fails when trying to import the base image (a tarball) due to a permissions error when calling vdirectory.import_files() in element.py

Stack trace

Here is the stack trace I received:

[--:--:--] START   bootstrap.bst: Staging sources
[--:--:--] BUG     bootstrap.bst: Build

    An unhandled exception occured:
    
    Traceback (most recent call last):
      File "/home/jennis/buildstream/buildstream/_scheduler/jobs/job.py", line 398, in _child_action
        result = self.child_process()
      File "/home/jennis/buildstream/buildstream/_scheduler/jobs/elementjob.py", line 94, in child_process
        return self._action_cb(self._element)
      File "/home/jennis/buildstream/buildstream/_scheduler/queues/buildqueue.py", line 70, in process
        element._assemble()
      File "/home/jennis/buildstream/buildstream/element.py", line 1516, in _assemble
        collect = self.assemble(sandbox)
      File "/home/jennis/buildstream/buildstream/plugins/elements/import.py", line 71, in assemble
        self._stage_sources_in_sandbox(sandbox, 'input', mount_workspaces=False)
      File "/home/jennis/buildstream/buildstream/element.py", line 1346, in _stage_sources_in_sandbox
        self._stage_sources_at(host_vdirectory, mount_workspaces=mount_workspaces)
      File "/home/jennis/buildstream/buildstream/element.py", line 1379, in _stage_sources_at
        vdirectory.import_files(temp_staging_directory)
      File "/usr/lib/python3.5/tempfile.py", line 947, in __exit__
        self.cleanup()
      File "/usr/lib/python3.5/tempfile.py", line 951, in cleanup
        _rmtree(self.name)
      File "/usr/lib/python3.5/shutil.py", line 474, in rmtree
        _rmtree_safe_fd(fd, path, onerror)
      File "/usr/lib/python3.5/shutil.py", line 412, in _rmtree_safe_fd
        _rmtree_safe_fd(dirfd, fullname, onerror)
      File "/usr/lib/python3.5/shutil.py", line 432, in _rmtree_safe_fd
        onerror(os.unlink, fullname, sys.exc_info())
      File "/usr/lib/python3.5/shutil.py", line 430, in _rmtree_safe_fd
        os.unlink(name, dir_fd=topfd)
    PermissionError: [Errno 13] Permission denied: 'ldconfig'

What's going on

This is what we're doing:

1. Staging sources in a temporary directory (here)
2. Then importing them from the temporary directory into a virtual directory (here)
3. This import_files() call eventually calls os.unlink(), which is semantically identical to os.remove().
4. import_files() calls copy_files() in utils (here).
5. copy_files() calls _process_list() which is used to copying/moving/linking file lists.

So, it looks like when we copy files from the temp dir, we're also deleting them. And we simply do not have the permission to delete these files.

I'm not sure what the best solution is for this, because my guess is that at some point, files in the tmp dir, or the tmp dir itself will need to be deleted, and we won't have the permission to do this.

Another thought is that it appears that _process_list() handles file permissions by setting them at the end, perhaps we're invoking this twice?

Any thoughts?

---