Skip to content

[SE-6050] fix: tracking logs format

Matjaz Gregoric requested to merge mtyaka/SE-6050-tracking-logs-format into main

Description

This uses some fluent-bit filters to strip off log metadata and only send the JSON part of tracking logs to S3.

It also rewrites the tracking log tag in a way that allows for logs from each instance to go into a separate folder, named after the instance k8s namespace._

Testing instructions

Steps to test the changes:

This update has been set up on the grove-stage-digitalocean cluster with a test S3 bucket called test-grove-stage-digitalocean-tracking-logs in the OpenCraft AWS account.

Log into the AWS console, find the bucket, verify that it contains tracking logs from instances from the cluster, and that the content of the files is in the expected format.

Screenshots

Screenshot_2023-10-27_at_09.04.39

Screenshot_2023-10-27_at_09.06.21

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

  • All providers include the new feature/change
  • All affected providers can provision new clusters
  • Unit tests are added/updated
  • Documentation is added/updated
  • The TOOLS_CONTAINER_IMAGE_VERSION in ci_vars.yml is updated
  • The grove-template repository is updated

Merge request reports