Manage kcidb subtests
Currently, we're generating subtests from two different places:
Bkr2kcidb uses a beaker file, and it includes some subtests added by Beaker, and the status of the test and results is consistent. Upt uses restraint in standalone mode, and the test status and results are inconsistent.
Bkr2kcidb does not manage log files. Currently, it only references beaker links for log files. There is an issue with uploading files to S3 without adding fake log files. Upt manages log files and it creates fake log files for special cases.
Bkr2kcidb adds all subtests even when the subtest does not have logs. Upt adds only some subtests without logs, adding a fake log file (when local watchdog and kernel happen).
When bkr2kcidb processes a subtest, the test status is taken into account, upt not.
We should define the rules for processing subtests.
As far as I know, we have some agreements:
- Test and subtest statuses should be related. The worst subtest status should be the status of the test.
- We should show as much information as possible. By default, any subtest should be submitted.
- We should change the path of subtests when the path is
/
with the text of the XML field. - In upt, we could add subtest information when restraint has any problem. The behavior should be similar to Beaker.
Time to discuss.
- Adding fake log files could be great for dw triaging, but the behavior should be consistent. Maybe we can create a list of fake log files in any s3 bucket. Upt and bkr2kcidb could link to those files instead of creating new ones. Maybe users are creating datawarehouse tickets and regular expressions.
Links: