[retried_tests.ml]: send bi-weekly reports via slack
Builds upon: !59 (merged)
To test it locally, make sure to first collect the metrics retried_tests.ml
.
Finally, you need to update local-sandbox/tezt_config.json
to set the webhooks per team. Here's an example config that can be used. It will send all the alerts to the #arvid-test-flake-report channel on the NL slack, feel free to spam it at your hearts content.
{
"influxdb": {
"url": "http://localhost:8086",
"database": "performance_regression",
"measurement_prefix": "",
"timeout": 20
},
"grafana": {
"url": "http://localhost:3000/api",
"data_source": "InfluxDB",
"timeout": 20
},
"alerts": {
"slack_webhook_urls": {
"default": "https://hooks.slack.com/services/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",
"infrastructure": "https://hooks.slack.com/services/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",
"layer1": "https://hooks.slack.com/services/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",
"tezos2": "https://hooks.slack.com/services/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",
"etherlink": "https://hooks.slack.com/services/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"
}
},
"test_data_path" : "/path/to/your/folder"
}
Then do:
TEZT_CONFIG=local-sandbox/tezt_config.json dune exec reports/main.exe -- -v -a retried_tests_report.limit=2
to send some reports.
You can also do:
TEZT_CONFIG=local-sandbox/tezt_config.json dune exec reports/main.exe -- -v -a retried_tests_report.limit=2 -a retried_tests_report.dry_run=true
to just send test reports instead.