Performance regression in `bst show` and `bst build`

https://mail.gnome.org/archives/buildstream-list/2019-November/msg00012.html

[11:13:30] <traveltissues> last week we seem to have lost about a second on show and 15 on building
[11:17:58] <traveltissues> casd was updated in the ci to resolve an api incompatibility though, so it's not clear if that's a reflection of buildstream performance
[11:18:55] <traveltissues> juergbi, is there any benchmarking done on buildbox?
[11:21:11] <juergbi> no benchmarking in buildbox-casd on its own
[11:21:35] <juergbi> likely related to https://gitlab.com/BuildStream/buildstream/issues/1126
[11:21:57] <juergbi> https://gitlab.com/BuildStream/buildstream/merge_requests/1642
[11:22:30] <juergbi> i.e., probably not a slowdown in buildbox-casd itself, rather how we use it
[11:22:40] <traveltissues> right
[11:22:57] <juergbi> and to some extent an issue with the not quite realistic synthetic test
[11:23:57] <traveltissues> the slowdown for build is ~10% and ~15% for show though
[11:24:32] <traveltissues> i assume this is not avoidable though
[11:25:33] <traveltissues> do you think this is worth looking into juergbi?
[11:27:45] <juergbi> traveltissues: batching and parallelizing the requests pre-scheduler should improve/fix the bst show slowdown
[11:27:49] <juergbi> and I think it's worth doing that
[11:28:42] <juergbi> the build time issue we might be able to fix by skipping the check in the main process after successful child job
[11:29:15] <juergbi> i.e., in the typical case where the child job says it has successfully added something to CAS (be it sources or artifacts), we shouldn't have to check this in the main process
[11:29:29] <juergbi> but rather simply trust the child job and mark it as cached in the main process
[11:29:42] <juergbi> and that's probably worth fixing as well