Apdex Metrics calculations appear to go blank if a single cluster is out of rotation

Problem Statement

During a recent maintenance to rollout gitlab-sshd, part of the procedure was to remove traffic to one cluster to perform the configuration change. After the configuration change is completed, we then tested the service, and only after confirmation that all is well we allowed traffic back onto that cluster for ServiceGitlab Shell. During the times for which a cluster was not receiving traffic, the Apdex panel of the Overview dashboard has a hole in our metrics. This is effectively creates a blind spot in our metrics.

image

Source

Make note that the chart above is showcasing our Main stage, and this is our git overview, which includes ServiceGitlab Shell and ServiceGit. The Canary chart for the same time does NOT have missing metrics.

Milestones

Leverage this issue to determine why the metrics have holes when there are at least 2 other clusters that are accepting and processing user traffic.