BuildStream refuses to cache artifacts with files that have very strict permissions

Summary

When trying to build an element that produces an artifact which contains a file without write permissions, BuildStream will fail to cache the artifact.

This is particularly annoying when trying to depend on a base platform with, say, a /etc/shadow file.

Steps to reproduce

1. Create a simple BuildStream project.

2. Create a tarfile containing a file with its permissions set to 400.

3. Create a .bst file similar to:

   ```yaml
   kind: import
   sources:
   - kind: tar
     base-dir: ''
     url: file:///src/src/permissions.tar.gz
   ```

4. Attempt to build the element.

What is the current bug behavior?

BuildStream will fail to cache the resulting files in an artifact.

What is the expected correct behavior?

BuildStream caches the resulting artifact.

Relevant logs and/or screenshots

The error message given when BuildStream attempts to cache:

[--:--:--] START   [readonly.bst] : Staging sources
[00:00:00] SUCCESS [readonly.bst] : Staging sources
[--:--:--] START   [readonly.bst] : Caching artifact
[00:00:00] FAILURE [readonly.bst] : Caching artifact
[00:00:00] FAILURE [readonly.bst] : Failed to commit artifact: openat(none): Permission denied
======================================================================

Other relevant information

• BuildStream version affected: /milestone %"BuildStream_v1.1"
• BuildStream version affected: /milestone %BuildStream_v1.2
• BuildStream version affected: /milestone %"BuildStream_v1.3"

---

changed the description

Because tar thinks it is allowed to do something if it is uid 0 ?

Or is this OSTree failing to commit readonly files ? I somehow doubt this, i think i've seen similar OSTree issues fixed.

@tristanvb not having had a much deeper look into this, I suspect it's on the OSTree side, since the source has already been checked out. The error happens when the artifact is committed to the cache.

This might also happen when we copy between temporary directories, or something, but I doubt it's an issue with tar, especially as I've seen it happen with docker and file sources as well.

marked this issue as related to bst-external#3 (moved)

mentioned in issue bst-external#3 (moved)

This appears to be fixed with the introduction of CAS :)

I'm not sure this can be backported, so I will close it for now.

closed

mentioned in merge request bst-external!40

removed To Do label

I've just hit this (or an issue very similar) when trying to use the CentOS OSTree repo -- usr/bin/ssh-agent is set to 0111 and causes BuildStream to error out.

@hdonnay did you get the exact same error message?

Re-opening.

reopened

No, just a similar permissions message. I've attached a reproducing setup; perhaps someone with more familiarity can determine if this is the same issue or a new one.

@hdonnay The error in your setup happens when staging sources, not when caching the artifact. It's either an issue in the OSTree source plugin or in BuildStream core, as far as I can tell. As it's not quite the same issue, please open a new one. I'm closing this again.

closed

Sorry, the logs printed for me seem to indicate it's the caching:

        [--:--:--] START   centos.bst: Build
        [--:--:--] LOG     centos.bst: Build environment for element centos.bst

            HOME: /tmp
            LC_ALL: C
            LOGNAME: tomjon
            PATH: /usr/bin:/bin:/usr/sbin:/sbin
            SHELL: /bin/sh
            SOURCE_DATE_EPOCH: '1320937200'
            TERM: dumb
            TZ: UTC
            USER: tomjon
            USERNAME: tomjon
        [--:--:--] START   centos.bst: Staging sources
        [--:--:--] START   centos.bst-0: Staging ref: b62513420243def54db7d6c36a79d8f0eda94c5ac410c1f64ee6e0b44e5dca99 from origin: http://mirror.centos.org/centos/7/atomic/x86_64/repo/
        [00:00:08] SUCCESS centos.bst-0: Staging ref: b62513420243def54db7d6c36a79d8f0eda94c5ac410c1f64ee6e0b44e5dca99 from origin: http://mirror.centos.org/centos/7/atomic/x86_64/repo/
        [00:00:08] SUCCESS centos.bst: Staging sources
        [--:--:--] START   centos.bst: Caching artifact
        [00:00:02] FAILURE centos.bst: Caching artifact
        [00:00:12] FAILURE centos.bst: Failed to hash object: [Errno 13] Permission denied: '/home/hank/.cache/buildstream/build/centos-upncikpw/artifact/files/usr/bin/ssh-agent'

BuildStream master already fails staging sources while 1.2 fails caching the artifact. I've opened #684 for the former issue (master only). The latter issue (also in 1.2) fits here.

reopened

mentioned in issue #684

marked this issue as related to #684

changed milestone to %BuildStream_v1.2

added Important To Do labels

mentioned in merge request !837 (closed)

changed milestone to %BuildStream_v1.4

marked this issue as related to #324

marked this issue as related to #38