Extend gstg watch to include NameError
See gitlab-org/gitlab#348698 (closed), and gitlab-com/gl-infra/production#6083 (closed)
Testing
I modified elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet to extend the range:
diff --git a/elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet b/elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet
index ac38a962..27ab33fa 100644
--- a/elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet
+++ b/elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet
@@ -16,7 +16,7 @@ local es_query = {
{
range: {
'@timestamp': {
- gte: std.format('now-%dm', query_period),
+ gte: std.format('now-%dd', query_period),
lte: 'now',
},
},
Then generated the JSON with:
jsonnet elastic/managed-objects/nonprod-log/watches/no_method_error.jsonnet
Simulating the watch in Elastic gave me:
"condition": {
"type": "compare",
"status": "success",
"met": true,
"compare": {
"resolved_values": {
"ctx.payload.hits.total": 71
}
}
},
"actions": [
{
"id": "notify-slack",
"type": "slack",
"status": "simulated",
"slack": {
"message": {
"from": "ElasticCloud Watcher: NoMethodError",
"to": [
"#staging"
],
"text": "NoMethodError: 71 errors detected!",
"attachments": [
{
"title": ":spiral_note_pad: NoMethodError in Rails logs:",
"text": "https://nonprod-log.gitlab.net/goto/28345690-61e0-11ec-aec7-f387696fcb42"
},
{
"title": ":spiral_note_pad: NoMethodError in Sidekiq logs:",
"text": "https://nonprod-log.gitlab.net/goto/3e0b0180-61e0-11ec-aec7-f387696fcb42"
},
{
"title": ":runbooks: Runbook:",
"text": "https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/uncategorized/staging-environment.md#elasticcloud-watcher-nomethoderror"
}
]
}
}
}
]
},
"messages": []
}
Which matches what I see in https://nonprod-log.gitlab.net/goto/28345690-61e0-11ec-aec7-f387696fcb42
Edited by Thong Kuah
