Tempest Pushgateway Support

Summary

Right now there are no metics for Tempest. If a test fails the pod terminates with an exit code != 0 and it is up to the kubernetes monitoring to catch the error. In order to make a more granular monitoring, a exporter should be implemented, that pushes the result of the tests to a prometheus pushgateway.

Use cases

• More granular alerting than failed or success based on the exit code
• Historical data to catch patterns (if possible)

Proposal

Implement a field in the CRD to specifie the target pushgateway. Use a subunit parser to get the results in a machine readable way. Push the results to the pushgateway via a small (e.g. Python) script.

Example Subunit Parser:

tempest run --config-file /home/tempest/etc/tempest.conf --regex {{ tempest_regex }} --subunit > /home/tempest/result/result.log 
cat /home/tempest/result/result.log | subunit2junitxml > /home/tempest/result/tests_results.xml

Specification

• MUST generate a metric tempest_failed_test for each failed test with the name as a label
• MUST generate the metrics tempest_test_count, tempest_successfull_tests_count, tempest_failed_tests_count and tempest_skipped_test_count
• SHOULD log the results in a human redable way, as the --subunit flag removes the nice result summary ath the end
• MUST add a optional pushgateway config to the CRD
• MUST only try pushing metrics, when the pushgateway config is specified
• MUST implement a tempest_last_run metric, with the timestamp of the current