Skip to content

Updates for support of ES 8 & OpenSearch

What does this Merge Request do and why?

There was a recent change to upgrade the rails dependency used to communicate with Elasticsearch. The new dependency throws warnings if Elasticsearch security is disabled because that would be a risk for production. However, this is the default behavior for Elasticsearch instances and what we want for gdk and development use. Also, it's pretty annoying when running rspec tests.

This MR changes:

  • disable security using the elasticsearch/config/elasticsearch.yml configuration file on gdk install and gdk reconfigure
  • Note: if security is already set in the yml file, this will not override that
  • updates documentation for how to run the now supported Elasticsearch v8.X and OpenSearch v1.X and removes mention of Elasticsearch 6.X which is not supported anymore

Log example

➜ bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
DEPRECATION WARNING: Database connection should not be called during initializers. Read more at https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers (called from connected? at ee/lib/gitlab/geo.rb:79)
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 9.235826 seconds
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."
warning: 299 Elasticsearch-7.16.1-5b38441b16b1ebb16a27c107a4c3865776e20c53 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security."

To test

I tested this in a working instance of gdk with Elasticsearch enabled and setup. You can run the spec mentioned above to verify the message happens. Not sure if there is a better way to test this out.

  • look at the elasticsearch config file
    cat ../elasticsearch/config/elasticsearch.yml
  • run the rspec test and verify you see the warning messages: bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
  • replicate the two file changes in the working instance of gdk
  • run the make command
    support/elasticsearch-config
  • look at the elasticsearch config file: cat ../elasticsearch/config/elasticsearch.yml
  • verify the config file has xpack.security.enabled: false at the end
  • restart elasticsearch gdk restart elasticsearch
  • run the rspec test and verify there are NO warning messages: bin/rspec ee/spec/workers/elastic/migration_worker_spec.rb
  • run the make command
    support/elasticsearch-config
  • look at the elasticsearch config file
    cat ../elasticsearch/config/elasticsearch.yml
  • verify the config file has only has one instance xpack.security.enabled: false at the end (we do not want to duplicate it every time reconfigure is run)

Merge Request checklist

  • This change is backward compatible. If not, please include steps to communicate to our users.
  • Tests added for new functionality. If not, please raise Issue to follow-up.
  • Documentation added/updated, if needed.
  • gdk doctor test added, if needed.
  • Add the ~highlight label if this MR should be included in the CHANGELOG.md.
Edited by Terri Chu

Merge request reports