Pass hash instead of URI to Elasticsearch client

What does this MR do and why?

We have issues when there are special characters in Elasticsearch credentials. Historically, we passed a URI to the elasticsearch client, but that forces us to url encode the credentials. This MR changes the Elasticsearch connection details to be passed to elasticsearch client as a hash to avoid this problem.

It's worth noting that the elasticsearch client gem uses a raw hash anyways: https://github.com/elastic/elasticsearch-ruby/blob/v7.3.0/elasticsearch-transport/lib/elasticsearch/transport/client.rb#L196

Additionally, this doesn't put any additional guard rails around passing credentials with special characters in elasticsearch_url application setting. Users still need to pass a URI safe url that won't raise an error in URI.parse. For example, this still will not work:

[1] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://user:p@ssword@localhost:9200/')
URI::InvalidURIError: bad URI(is not URI?): "http://user:p@ssword@localhost:9200"

It is expected that the user either a) percent-encode the credentials in the URL or b) use raw, unencoded values in elasticsearch_username and elasticsearch_password application settings:

[2] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://user:p%40ssword@localhost:9200/', elasticsearch_username: nil, elasticsearch_password: nil)
=> true
[3] pry(main)> ApplicationSetting.current_without_cache.elasticsearch_url_with_credentials
=> [{:scheme=>"http", :user=>"user", :password=>"p@ssword", :host=>"localhost", :path=>"", :port=>9200}]
[4] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://localhost:9200/', elasticsearch_username: 'user', elasticsearch_password: 'newp@ssword')
=> true
[5] pry(main)> ApplicationSetting.current_without_cache.elasticsearch_url_with_credentials
=> [{:scheme=>"http", :user=>"user", :password=>"newp@ssword", :host=>"localhost", :path=>"", :port=>9200}]

Related to: #13739 (closed) Closes: #354456 (closed)

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

  1. If you have gdk Elasticsearch running, do gdk stop elasticsearch
  2. Start an elasticsearch container with password that includes a special character:
docker run --rm -p 9200:9200 -e "discovery.type=single-node" -e 'xpack.security.enabled=true' -e 'ELASTIC_PASSWORD=p@ssword' docker.elastic.co/elasticsearch/elasticsearch:7.16.1
  1. In a rails console, update the advanced search settings
ApplicationSetting.current.update(elasticsearch_url: 'http://localhost:9200/', elasticsearch_username: 'elastic', elasticsearch_password: 'p@ssword')
  1. Run indexing rake task: bundle exec rake gitlab:elastic:index. This is asynchronous and may take a bit of time to complete even after command exits successfully.
  2. Verify that code was indexed successfully by visiting http://localhost:9200/_search?q=flight

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by John Mason

Merge request reports

Loading