Skip to content

Mangled symlinks when importing freedesktop BasePlatform

Summary

When testing bst-external locally against buildstream master, I get the error "FAILURE base/base-system.bst: Staged artifacts do not provide command 'sh'". Full text at 1.

On closer investigation, in the source cache /bin/sh is a symlink to /bin/bash 2, but in the artifact cache /bin/sh is a symlink to ../../bin/bash 3.

Steps to reproduce

  • In bst-external, go to tests/project and run bst build base.bst
    • NOTE: because of some other current issues, you may need to go into project.conf and delete everything in the plugins: section.

What is the current bug behavior?

  • A symlink from /bin/sh to /bin/bash becomes a symlink to ../../bin/bash
  • Sandbox commands that use that BasePlatform fail because there's no sh

What is the expected correct behavior?

  • A symlink from /bin/sh to /bin/bash becomes a symlink to ../bin/bash (or just bash
  • Sandbox commands that use the BasePlatform continue to run commands as normal

Relevant logs and/or screenshots

Possible fixes

I don't know where the fault lies (though the code to turn absolute symlinks into relative ones is in utils.py:866 "_relative_symlink_target").

I do know when it happened, this behaviour occurs after commit "eabc3899", "Merge branch 'jmac/virtual_directories' into 'master'.

Notably, bst-1.2 behaves perfectly fine.

Other relevant information

  1. The build output is as follows:

    [--:--:--][][] START   Build
[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources

BuildStream Version 1.3.0+319.g255f9ee3
  Session Start: Friday, 24-08-2018 at 12:38:22
  Project:       test (/home/jonathanmaw/workspace/buildstream/bst-external/tests/project)
  Targets:       base.bst

User Configuration
  Configuration File:      /home/jonathanmaw/.config/buildstream.conf
  Log Files:               /media/jonathanmaw/external/buildstream-cache-5/logs
  Source Mirrors:          /media/jonathanmaw/external/buildstream-cache-5/sources
  Build Area:              /media/jonathanmaw/external/buildstream-cache-5/build
  Artifact Cache:          /media/jonathanmaw/external/buildstream-cache-5/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Project Options
  arch:  x86_64
  linux: 1

Pipeline
fetch needed 2526aad094f4a68f2dda3027807615960e61271eabfd12b181c441530cd7efc5 base/base-platform.bst 
     waiting 262f402bb7c3324f4ff0e3f032e39116080ab12238dbff4346d61d13b4f6cd8d base/base-system.bst 
     waiting b910b9f9162f23240ab865c4356046355833b8e5e2f911b944ea6616675141a2 base/base-configure.bst 
     waiting f74d6a152b932f80b0d8653b94b892da98557038b013c8bf7f2c1bda544ab26e base.bst 
===============================================================================
[--:--:--][2526aad0][fetch:base/base-platform.bst        ] START   test/base-base-platform/2526aad0-fetch.23775.log
[--:--:--][2526aad0][fetch:base/base-platform.bst        ] START   Fetching remote ref: c9d09b7250a12ef09d95952fc4f49a35e5f8c2c1dd7141b7eeada4069e6f6576 from origin: https://sdk.gnome.org/repo/
[--:--:--][2526aad0][fetch:base/base-platform.bst        ] STATUS  Receiving objects: 42% (3/7) 676 bytes
[--:--:--][2526aad0][fetch:base/base-platform.bst        ] STATUS  Receiving objects: 2% (27/1195) 70.5 kB
---------------------------------------------------->8-SNIP->8---------------------------------------------
[--:--:--][2526aad0][fetch:base/base-platform.bst        ] STATUS  Receiving objects: 99% (10154/10159) 99.9 MB
[00:03:21][2526aad0][fetch:base/base-platform.bst        ] SUCCESS Fetching remote ref: c9d09b7250a12ef09d95952fc4f49a35e5f8c2c1dd7141b7eeada4069e6f6576 from origin: https://sdk.gnome.org/repo/
[00:03:24][2526aad0][fetch:base/base-platform.bst        ] SUCCESS test/base-base-platform/2526aad0-fetch.23775.log
[--:--:--][2526aad0][build:base/base-platform.bst        ] START   test/base-base-platform/2526aad0-build.30201.log
[--:--:--][2526aad0][build:base/base-platform.bst        ] START   Staging sources
[00:00:16][2526aad0][build:base/base-platform.bst        ] SUCCESS Staging sources
[--:--:--][2526aad0][build:base/base-platform.bst        ] START   Caching artifact
[00:00:07][2526aad0][build:base/base-platform.bst        ] SUCCESS Caching artifact
[00:00:29][2526aad0][build:base/base-platform.bst        ] SUCCESS test/base-base-platform/2526aad0-build.30201.log
[--:--:--][262f402b][build:base/base-system.bst          ] START   test/base-base-system/262f402b-build.32108.log
[--:--:--][262f402b][build:base/base-system.bst          ] START   Staging dependencies
[00:00:00][262f402b][build:base/base-system.bst          ] SUCCESS Staging dependencies
[--:--:--][262f402b][build:base/base-system.bst          ] START   Integrating sandbox
[--:--:--][262f402b][build:base/base-system.bst          ] STATUS  Running integration command

    ldconfig

[00:00:00][262f402b][build:base/base-system.bst          ] FAILURE Integrating sandbox
[--:--:--][262f402b][build:base/base-system.bst          ] START   Caching artifact
[00:00:00][262f402b][build:base/base-system.bst          ] SUCCESS Caching artifact
[00:00:00][262f402b][build:base/base-system.bst          ] FAILURE Staged artifacts do not provide command 'sh'

    Sandbox directory: /media/jonathanmaw/external/buildstream-cache-5/build/base-base-system-gkx_is5m

    Printing the last 20 lines from log file:
    /media/jonathanmaw/external/buildstream-cache-5/logs/test/base-base-system/262f402b-build.32108.log
    ======================================================================
        PATH: /usr/bin:/bin:/usr/sbin:/sbin
        SHELL: /bin/sh
        SOURCE_DATE_EPOCH: '1320937200'
        TERM: dumb
        TZ: UTC
        USER: tomjon
        USERNAME: tomjon
        V: '1'
    [--:--:--] START   base/base-system.bst: Staging dependencies
    [--:--:--] START   base/base-platform.bst: Staging base/base-platform.bst/2526aad0
    [00:00:00] SUCCESS base/base-platform.bst: Staging base/base-platform.bst/2526aad0
    [00:00:00] SUCCESS base/base-system.bst: Staging dependencies
    [--:--:--] START   base/base-system.bst: Integrating sandbox
    [--:--:--] STATUS  base/base-platform.bst: Running integration command

        ldconfig
    [00:00:00] FAILURE base/base-system.bst: Integrating sandbox
    [--:--:--] START   base/base-system.bst: Caching artifact
    [00:00:00] SUCCESS base/base-system.bst: Caching artifact
    [00:00:00] FAILURE base/base-system.bst: Staged artifacts do not provide command 'sh'
    ======================================================================
    ↩

  2. When inspecting the BasePlatform in the source cache

    jonathanmaw@fafnir:~/external/buildstream-cache-5/sources/ostree/gnomesdk_repo_$ ostree ls c9d09b7250a12ef09d95952fc4f49a35e5f8c2c1dd7141b7eeada4069e6f6576 files/bin/sh
l00777 0 0      0 /files/bin/sh -> /bin/bash
    ↩

  3. When inspecting the BasePlatform in the artifact cache

    jonathanmaw@fafnir:~/external/buildstream-cache-5/artifacts/extract/test/base-base-platform/686c11b1e91f03923998c3cb1f244ac8637051d197dbcb5bed1611d790734ac1$ ls -l files/bin/sh
lrwxrwxrwx 1 jonathanmaw jonathanmaw 14 Aug 24 12:42 files/bin/sh -> ../../bin/bash
    ↩
Edited by Jonathan Maw
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information