The command-line should provide access to artifacts

Background

BuildStream doesn't provide an interface to its artifact cache directly.

This hasn't been sufficient and has led to error-prone manual access to ~/.cache/buildstream.

See https://mail.gnome.org/archives/buildstream-list/2018-July/msg00051.html for the source discussion of this issue.

#234 (closed) is related in that this would provide an interface for reading logs in addition to its other requirements.

Acceptance Criteria

• There is a bst artifact subcommand group for interacting with the cache.

• bst artifact subcommands take artifact ref names.

• Artifact ref names can be tab-completed.

• A glob expression may be used instead of a name.

• An element name may be used instead of an artifact ref name.

• There are bst artifact subcommands to replace bst checkout, bst pull and bst push.

• There is a bst artifact list command for listing existing artifacts, which takes a --long/-l option for more information as described in #416 (comment 83263225).

  Artifacts are listed in order of build date, display the build date in local time, list the size of the files portion of the artifact, and the name of the active workspace if it was created from a local workspace build.

• There is a bst artifact log command for viewing logs in a pager.

• There is a bst artifact delete command for deleting artifacts that match the specified pattern.

• There is a bst artifact list-content command for listing the contents of an artifact. It should take a --long/-l option to show more information, like ls -l or tar -t [-f] -vv.

• There is a bst artifact diff command for showing which files were added/removed/changed or are the same between artifacts.

---

For 1.4 the new bst artifact checkout command won't support all the options of bst checkout for artifact ref name arguments, but can still be used as a replacement for bst checkout when passed element names.

See https://mail.gnome.org/archives/buildstream-list/2018-November/msg00097.html for discussion.

mentioned in merge request !1008 (closed)

marked this issue as related to #477 (closed)

mentioned in issue #477 (closed)

mentioned in merge request !1049 (closed)

mentioned in merge request !1060 (closed)

UPDATE:

• !1008 (closed) was introduced to address most of the check boxes in the description of this issue. From this !1049 (closed) and !1060 (closed) were introduced to split out from !1008 (closed) in order to land artifact delete and bst artifact checkout separately with appropriate tests.

• !1045 (merged) was also introduced to move push/pull/checkout to the artifact sub command group however they can currently only support element names and not artifact refs (which they should be able too).

• The reason many of these have not landed is because after some discussion on IRC and at the BuildStream gatherings it was agreed that the earlier MRs bleed out too much low level logic in the front end. These discussions were summarised on the mailing list (here). Because of this, a refactor was required before any more work on the artifact subcommands, this refactor has now landed: !1101 (merged).

• !1101 (merged) has ensured that we can load artifact refs as well as element names. It has also introduced a new ArtifactElement class which inherits from Element.

• Now, !1049 (closed) and !1060 (closed) need to be rebased/reworked so that they work with the refactor (!1101 (merged)). Delete shouldn't be so hard.

• I think it's important that we first try to have bst artifact checkout/pull/push handle/support artifact refs before working on the other artifact subcommands discussed in this issue's description and attempted in !1008 (closed).

mentioned in issue #822 (closed)

Relevant ML post from a few weeks back - https://mail.gnome.org/archives/buildstream-list/2019-June/msg00004.html

mentioned in issue #1068

changed the description

marked the checklist item There is a bst artifact log command for viewing logs in a pager. as completed

marked the checklist item There is a bst artifact delete command for deleting artifacts that match the specified pattern. as completed

marked the checklist item There are bst artifact subcommands to replace bst checkout, bst pull and bst push. as completed

marked the checklist item There is a bst artifact subcommand group for interacting with the cache. as completed

marked the checklist item bst artifact subcommands take artifact ref names. as completed

marked the checklist item Artifact ref names can be tab-completed. as completed

marked the checklist item Artifact ref names can be tab-completed. as incomplete

marked the checklist item An element name may be used instead of an artifact ref name. as completed

marked the checklist item bst artifact subcommands take artifact ref names. as incomplete

assigned to @beckyella16 and unassigned @richardmaw-codethink

mentioned in merge request !1497 (merged)

mentioned in issue #1091

marked the checklist item bst artifact subcommands take artifact ref names. as completed

closed via merge request !1497 (merged)

mentioned in commit 4b23d444

Grr, how is this closing-accidentally-with-messages thing still an open bug, GitLab?

reopened

marked the checklist item Artifact ref names can be tab-completed. as completed

marked the checklist item A glob expression may be used instead of a name. as completed

We currently have both tab completion and glob expression matching for artifact ref's, so I ticked these 2 off. We also have tests for it here and here for artifact log.

@jennis suggested it would be a good a idea to also have a test for artifact delete using a glob expression, so I will try to do that.

LE: As I have just started working on another project, it seems like I can't be doing this, so if anyone fancies adding a test for artifact delete *, that would be awesome!

mentioned in merge request !1529 (merged)

mentioned in merge request !1555 (merged)

closed via merge request !1555 (merged)

mentioned in commit 41c09fbe

marked the checklist item There is a bst artifact list-content command for listing the contents of an artifact. as completed

reopened

mentioned in issue #355 (closed)

marked the checklist item There is a bst artifact list command for listing existing artifacts, which takes a --long/-l option for more information as described in #416 (comment 83263225). as completed

marked the checklist item There is a bst artifact list command for listing existing artifacts, which takes a --long/-l option for more information as described in #416 (comment 83263225). as incomplete

mentioned in issue #416 (closed)

This is mostly implemented, I believe we should handle additional enhancements separately from now on.

closed

changed the description