send kubelet logs to Elastic
As per comment here: delivery#738 (comment 410479871) we want to forward kubelet logs to Elastic. See comments below for a discussion on different approaches
-
nonprod kubelet logs: -
TF MR: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2118 -
enable dynamic mappings for GKE indices: gitlab-com/runbooks!2825 (merged) -
roll-over indices so that an index is created with dynamic enabled -
fix CI job: gitlab-com/runbooks!2826 (merged) -
retrigger CI job: gitlab-com/runbooks!2827 (merged) -
roll-over indices again -
configure static mappings in the runbooks repo anb disable dynamic mappings in the template: gitlab-com/runbooks!2829 (merged) -
revert static mappings due to a bug in index template upload script: gitlab-com/runbooks!2831 (merged) -
fix issue with CI jobs: gitlab-com/runbooks!2847 (diffs) -
readd gke mappings: gitlab-com/runbooks!2848 (merged) -
roll over indices for the changes to take effect -
all kubelet logs disappeared from the logging cluster, reverting static mappings: gitlab-com/runbooks!2851 (merged) -
reenable static mappings: gitlab-com/runbooks!2866 (merged) -
remove index_mappings_test
from nonprod cluster -
split indices -
ES config (runbooks, noprod and prod): gitlab-com/runbooks!2874 (merged) -
tf: clean up tf module: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2154 -
tf: add support for multiple filters, split the GKE topic: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2156 -
initialize indices in gstg (manual step) -
pubsubbeats for gstg: gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!254 (merged) -
fix IAM binding of the log writer accounts: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2157 -
fix filter expressions in gstg: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2158 -
confirm everything is working as desired: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/11452#note_435194533 -
static mappings for each index (except GKE): gitlab-com/runbooks!2877 (merged) -
roll over GKE index (manual step) -
upload new mappings based on dynamic ones: gitlab-com/runbooks!2878 (merged) -
ES: initialize indices in other envs (manual step) -
check for errors in the beat: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/11452#note_436559322 -
tf: roll out to other envs: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2159 -
pubsubbeats for other envs: gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!255 (merged) -
index patterns for pre and ops
-
-
-
gprd [ ] TF MR: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2117-
ES: initialize missing aliases (manual step) -
tf: switch to new module, split the GKE topic: https://ops.gitlab.net/gitlab-com/gitlab-com-infrastructure/-/merge_requests/2167 -
pubsubbeats (k8s config): gitlab-com/gl-infra/k8s-workloads/gitlab-helmfiles!256 (merged) -
ES: roll over GKE index (manual step) -
index patterns (manual step)
-
(optional) clean up PubSubs: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/11711
Edited by Michal Wasilewski