Skip to content

[retried_tests.ml]: send bi-weekly reports via slack

Arvid Jakobsson requested to merge arvid@flake-report-report into master

Builds upon: !59 (merged)

To test it locally, make sure to first collect the metrics retried_tests.ml.

Finally, you need to update local-sandbox/tezt_config.json to set the webhooks per team. Here's an example config that can be used. It will send all the alerts to the #arvid-test-flake-report channel on the NL slack, feel free to spam it at your hearts content.

{
    "influxdb": {
        "url": "http://localhost:8086",
        "database": "performance_regression",
        "measurement_prefix": "",
        "timeout": 20
    },
    "grafana": {
        "url": "http://localhost:3000/api",
        "data_source": "InfluxDB",
        "timeout": 20
    },
    "alerts": {
        "slack_webhook_urls": {
            "default": "https://hooks.slack.com/services/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",
            "infrastructure": "https://hooks.slack.com/services/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",
            "layer1": "https://hooks.slack.com/services/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",
            "tezos2": "https://hooks.slack.com/services/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",
            "etherlink": "https://hooks.slack.com/services/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"
        }
    },
    "test_data_path" : "/path/to/your/folder"
}

Then do:

TEZT_CONFIG=local-sandbox/tezt_config.json dune exec reports/main.exe -- -v -a retried_tests_report.limit=2

to send some reports.

You can also do:

TEZT_CONFIG=local-sandbox/tezt_config.json dune exec reports/main.exe -- -v -a retried_tests_report.limit=2 -a retried_tests_report.dry_run=true

to just send test reports instead.

Edited by Arvid Jakobsson

Merge request reports