Update ES client to 7.13.3, deprecate _type
What does this MR do and why?
Services Impacted
Advanced Search (backed by Elasticsearch). We are updating the Elasticsearch client which enables communication to the Elasticsearch server. This change will impact searching and indexing for new/updated data.
Note: This change is a breaking change that will remove support for Elasticsearch 6.8. We run Elasticsearch 7.X on GitLab.com so we do not anticipate this causing any compatibility issues.
What would be the mitigation path if something fails during the deployment of that MR?
Validation for searching and indexing will need to occur in staging. If issues are observed, an incident will be opened to prevent the production deployment. There is no sidekiq in gprd-cny
so only searches would be affected not indexing of data when the code is deployed to gprd-cny
. We would rely upon indexing validation in staging.
Any issues found would require a reverting the commit, which would fall back to using the existing ES client.
DRI Support
EMEA - @dgruzd AMER - @john-mason , @terrichu
Key metrics to observe
- Metric: Global Search errors
- Location: https://log.gprd.gitlab.net/goto/6cbe2100-c4bf-11ec-b73f-692cc1ae8214
- What changes to this metric should prompt a rollback: Errors related to client upgrade, we would look for errors related to connecting to Elasticsearch servers.
- Metric: Search overview metrics
- Location: https://dashboards.gitlab.net/d/search-main/search-overview?orgId=1 Hide charts
- What changes to this metric should prompt a rollback: Flatline of RPS
- Metric: Search controller performance
- Location: https://dashboards.gitlab.net/d/web-rails-controller/web-rails-controller?orgId=1&var-PROMETHEUS_DS=Global&var-environment=gprd&var-stage=main&var-controller=SearchController&var-action=show Hide charts
- What changes to this metric should prompt a rollback: Massive spike in latency
- Metric: Search sidekiq indexing queues (
Sidekiq Queues (Global Search)
)- Location: https://dashboards.gitlab.net/d/sidekiq-main/sidekiq-overview?orgId=1 Hide charts
- What changes to this metric should prompt a rollback: Queues not draining
- Metric: Search sidekiq in flight jobs
- Location: https://dashboards.gitlab.net/d/sidekiq-shard-detail/sidekiq-shard-detail?orgId=1&from=now-30m&to=now&var-PROMETHEUS_DS=Global&var-environment=gprd&var-stage=main&var-shard=elasticsearch Hide charts
- What changes to this metric should prompt a rollback: No jobs in flight
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #299769 (closed)