Pass hash instead of URI to Elasticsearch client
What does this MR do and why?
We have issues when there are special characters in Elasticsearch credentials. Historically, we passed a URI to the elasticsearch client, but that forces us to url encode the credentials. This MR changes the Elasticsearch connection details to be passed to elasticsearch client as a hash to avoid this problem.
It's worth noting that the elasticsearch client gem uses a raw hash anyways: https://github.com/elastic/elasticsearch-ruby/blob/v7.3.0/elasticsearch-transport/lib/elasticsearch/transport/client.rb#L196
Additionally, this doesn't put any additional guard rails around passing credentials with special characters in elasticsearch_url application setting. Users still need to pass a URI safe url that won't raise an error in URI.parse. For example, this still will not work:
[1] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://user:p@ssword@localhost:9200/')
URI::InvalidURIError: bad URI(is not URI?): "http://user:p@ssword@localhost:9200"
It is expected that the user either a) percent-encode the credentials in the URL or b) use raw, unencoded values in elasticsearch_username and elasticsearch_password application settings:
[2] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://user:p%40ssword@localhost:9200/', elasticsearch_username: nil, elasticsearch_password: nil)
=> true
[3] pry(main)> ApplicationSetting.current_without_cache.elasticsearch_url_with_credentials
=> [{:scheme=>"http", :user=>"user", :password=>"p@ssword", :host=>"localhost", :path=>"", :port=>9200}]
[4] pry(main)> ApplicationSetting.current.update(elasticsearch_url: 'http://localhost:9200/', elasticsearch_username: 'user', elasticsearch_password: 'newp@ssword')
=> true
[5] pry(main)> ApplicationSetting.current_without_cache.elasticsearch_url_with_credentials
=> [{:scheme=>"http", :user=>"user", :password=>"newp@ssword", :host=>"localhost", :path=>"", :port=>9200}]
Related to: #13739 (closed) Closes: #354456 (closed)
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
- If you have gdk Elasticsearch running, do
gdk stop elasticsearch - Start an elasticsearch container with password that includes a special character:
docker run --rm -p 9200:9200 -e "discovery.type=single-node" -e 'xpack.security.enabled=true' -e 'ELASTIC_PASSWORD=p@ssword' docker.elastic.co/elasticsearch/elasticsearch:7.16.1
- In a rails console, update the advanced search settings
ApplicationSetting.current.update(elasticsearch_url: 'http://localhost:9200/', elasticsearch_username: 'elastic', elasticsearch_password: 'p@ssword')
- Run indexing rake task:
bundle exec rake gitlab:elastic:index. This is asynchronous and may take a bit of time to complete even after command exits successfully. - Verify that code was indexed successfully by visiting http://localhost:9200/_search?q=flight
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.