Speed up `bst shell` command

bst shell is currently much too slow. It would be nice to aggressively attempt to speed things up. I know this is never going to be instantaneous, like jhbuild shell, but I hope we we can do a lot better here.

For instance:

$ bst shell --build core/epiphany.bst
[--:--:--][][] START   Loading pipeline
[00:00:00][][] SUCCESS Loading pipeline
[--:--:--][][] START   Resolving pipeline
[00:00:00][][] SUCCESS Resolving pipeline
[--:--:--][][] START   Resolving cached state
[00:01:35][][] SUCCESS Resolving cached state
[--:--:--][][] START   Staging dependencies
[--:--:--][????????][ main:core/epiphany.bst             ] WARNING Non-whitelisted overlaps detected

    Staged files overwrite existing files in staging area:
    /usr/bin/gtkdoc-check: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-depscan: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-fixxref: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-mkdb: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-mkhtml: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-mkman: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-mkpdf: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-rebase: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-scan: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdoc-scangobj: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/bin/gtkdocize: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/lib/cmake/GtkDoc/GtkDocConfig.cmake: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/lib/cmake/GtkDoc/GtkDocConfigVersion.cmake: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/lib/cmake/GtkDoc/GtkDocScanGObjWrapper.cmake: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/share/aclocal/gtk-doc.m4: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/share/gtk-doc/data/devhelp2.xsd: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/share/gtk-doc/data/devhelp2.xsl: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/share/gtk-doc/data/gtk-doc.flat.make: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    /usr/share/gtk-doc/data/gtk-doc.make: core-deps/gtk-doc.bst is not permitted to overlap other elements, order core-deps/gtk-doc.bst above base/base-configure.bst 
    Message contains 6766 additional lines

[00:00:12][][] SUCCESS Staging dependencies
[--:--:--][][] START   Integrating sandbox
[--:--:--][593e35e8][ main:base.bst                      ] STATUS  Running integration command

    ldconfig

[--:--:--][593e35e8][ main:base.bst                      ] STATUS  Running integration command

    PKGSYSTEM_ENABLE_FSYNC=0 update-mime-database /usr/share/mime

[--:--:--][593e35e8][ main:base.bst                      ] STATUS  Running integration command

    update-desktop-database -v

Search path is now: [/usr/local/share/applications, /usr/share/applications]
Could not create cache file in "/usr/local/share/applications": Error opening directory ?/usr/local/share/applications?: No such file or directory
File "/usr/share/applications/gtk3-widget-factory.desktop" lacks MimeType key
File "/usr/share/applications/gcr-prompter.desktop" lacks MimeType key
File "/usr/share/applications/python3.6.desktop" lacks MimeType key
File "/usr/share/applications/geoclue-where-am-i.desktop" lacks MimeType key
File "/usr/share/applications/geoclue-demo-agent.desktop" lacks MimeType key
File "/usr/share/applications/gtk3-demo.desktop" lacks MimeType key
File "/usr/share/applications/texdoctk.desktop" lacks MimeType key
File "/usr/share/applications/gtk3-icon-browser.desktop" lacks MimeType key
File "/usr/share/applications/python2.7.desktop" lacks MimeType key
[--:--:--][ec9dc658][ main:core-deps/glib.bst            ] STATUS  Running integration command

    glib-compile-schemas /usr/share/glib-2.0/schemas

Warning: Schema “org.gnome.crypto.pgp” has path “/desktop/gnome/crypto/pgp/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.locale” has path “/system/locale/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy” has path “/system/proxy/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.http” has path “/system/proxy/http/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.https” has path “/system/proxy/https/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.ftp” has path “/system/proxy/ftp/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
Warning: Schema “org.gnome.system.proxy.socks” has path “/system/proxy/socks/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
[--:--:--][81d1654d][ main:core-deps/gdk-pixbuf.bst      ] STATUS  Running integration command

    gdk-pixbuf-query-loaders --update-cache

[--:--:--][0566cf06][ main:core-deps/gtk+-3.bst          ] STATUS  Running integration command

    for dir in /usr/share/icons/*; do
      if test -f $dir/index.theme; then
        gtk-update-icon-cache --quiet $dir
      fi
    done

[--:--:--][0566cf06][ main:core-deps/gtk+-3.bst          ] STATUS  Running integration command

    gtk-query-immodules-3.0 --update-cache

[00:00:14][][] SUCCESS Integrating sandbox
[--:--:--][][] START   Staging sources
[00:00:00][][] SUCCESS Staging sources
[--:--:--][????????][ main:core/epiphany.bst             ] STATUS  Running command

    bash --noprofile --norc -i

Here we spent:

  • 1m35s resolving cached state
  • 12s staging dependencies
  • 14s integrating sandbox

Looks like "resolving cached state" is clearly the first thing to focus on.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information