Create HA proxy traffic SSoT in SiSense by service

Break out data transfer by service, tier, and whether this is Runner sourced. This would allow us to have insight into gitaly vs. other types of traffic, instead of it all being lumped into HA Proxy.

https://dashboards.gitlab.net/d/general-egress_ingress/general-network-ingress-egress-overview?viewPanel=23&orgId=1&refresh=5m

It is likely not possible to add this intelligence to HA Proxy, and so we need to move further up the application stack.

Needed data

We would like to be able to understand traffic utilization by:

  1. Namespace (Customer A, B, C) - This helps understand usage patterns by customer, and if there are any outliers we can work with them to reduce usage.
  2. Tier (Paid, Free, Internal) #13093 - This is important so we can correctly attribute the costs. This can be derived from namespace within the data warehouse if needed.
  3. Source (Runner, non-Runner) https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/13089 - This helps us understand how much of our transfer comes from CI. Knowing this can help us in our pricing efforts, as well as understanding usage mix.
Edited by Mek Stittri