ElasticSearch setup errors
I am not sure if I have problems or not with my ElasticSearch setup. Perhaps you might be able to interpret.
Our env:
Gitlab 8.16
Java 1.8.x (oracle)
ElasticSearch v2.4.4 with required ‘delete-by-query’ plugin
ElasticSearch is enabled both in Admin >> Settings, and in GitLab.rb file to use port 9200 and ‘localhost’
ElasticSearch Indexing is also enabled (ran gitlab-ctl reconfigure after all config changes)
I ran the create index command successfully:
gitlab-rake gitlab:elastic:create_empty_index
Then I ran the following:
gitlab-rake gitlab:elastic:clear_index_status
I get some errors (I could only post what was in my xterm buffer) – along with some indexing of content that looks successful (see very bottom). Are these errors a problem?
Also – how exactly is the ElasticSearch content consumed by Gitlab users/customers? Is this ES content available to the GitLab Search API?
<snip>…..
Indexing OWR / jira-test (ID=49)...
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:146:in `__raise_transport_error': [4 (Elasticsearch::Transport::Transport::Errors::NotFound)
<TITLE>Network Error</TITLE>
</HEAD>
<BODY>
<FONT face="Helvetica">
<big><strong></strong></big><BR>
</FONT>
<blockquote>
<TABLE border=0 cellPadding=1 width="80%">
<TR><TD>
<FONT face="Helvetica">
<big>Network Error (dns_unresolved_hostname)</big>
<BR>
<BR>
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
Your requested host "localhost" could not be resolved by DNS.
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica" SIZE=2>
<BR>
For assistance, contact your network support team.
</FONT>
</TD></TR>
</TABLE>
</blockquote>
</FONT>
</BODY></HTML>
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:256:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/client.rb:125:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:87:in `bulk'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:122:in `perform_bulk'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:210:in `block in index_commits'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `each'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `each_slice'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `index_commits'
from /opt/gitlab/embedded/service/gitlab-rails/bin/elastic_repo_indexer:61:in `block in <main>'
from /opt/gitlab/embedded/lib/ruby/2.3.0/benchmark.rb:293:in `measure'
from /opt/gitlab/embedded/service/gitlab-rails/bin/elastic_repo_indexer:59:in `<main>'
, trace - ["/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/elastic/indexer.rb:36:in `run'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:42:in `block (4 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in
each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in `block in find_each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:124:in `find_in_batches'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:50:in `find_each'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:26:in `block (3 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:7:in `block (3 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb100in `top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'", "/opt/gitlab/embedded/bin/rake:22:in `load'", "/opt/gitlab/embedded/bin/rake:22:in `<top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `load'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `kernel_load'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:27:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:332:in `exec'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:20:in `dispatch'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:11:in `start'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:34:in `block in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/friendly_errors.rb100in `with_friendly_errors'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:26:in `<top (required)>'", "/opt/gitlab/embedded/bin/bundle:22:in `load'", "/opt/gitlab/embedded/bin/bundle:22:in `<main>'"]
Indexing Retinskaya, Natalia / cc_bec_linux (ID=51)...
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:146:in `__raise_transport_error': [4 (Elasticsearch::Transport::Transport::Errors::NotFound)
<TITLE>Network Error</TITLE>
</HEAD>
<BODY>
<FONT face="Helvetica">
<big><strong></strong></big><BR>
</FONT>
<blockquote>
<TABLE border=0 cellPadding=1 width="80%">
<TR><TD>
<FONT face="Helvetica">
<big>Network Error (dns_unresolved_hostname)</big>
<BR>
<BR>
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
Your requested host "localhost" could not be resolved by DNS.
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica">
</FONT>
</TD></TR>
<TR><TD>
<FONT face="Helvetica" SIZE=2>
<BR>
For assistance, contact your network support team.
</FONT>
</TD></TR>
</TABLE>
</blockquote>
</FONT>
</BODY></HTML>
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:256:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/client.rb:125:in `perform_request'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-api-1.0.15/lib/elasticsearch/api/actions/bulk.rb:87:in `bulk'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:122:in `perform_bulk'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:210:in `block in index_commits'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `each'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `each_slice'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/gitlab-elasticsearch-git-1.0.1/lib/elasticsearch/git/repository.rb:205:in `index_commits'
from /opt/gitlab/embedded/service/gitlab-rails/bin/elastic_repo_indexer:61:in `block in <main>'
from /opt/gitlab/embedded/lib/ruby/2.3.0/benchmark.rb:293:in `measure'
from /opt/gitlab/embedded/service/gitlab-rails/bin/elastic_repo_indexer:59:in `<main>'
, trace - ["/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/elastic/indexer.rb:36:in `run'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:42:in `block (4 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:51:in `block in find_each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:124:in `find_in_batches'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/batches.rb:50:in `find_each'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:26:in `block (3 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'", "/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/elastic.rake:7:in `block (3 levels) in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'", "/opt/gitlab/embedded/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_wi
th_call_chain'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:150:in `invoke_task'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block (2 levels) in top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `each'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:106:in `block in top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:115:in `run_with_threads'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb100in `top_level'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:78:in `block in run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:176:in `standard_exception_handling'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/lib/rake/application.rb:75:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-10.5.0/bin/rake:33:in `<top (required)>'", "/opt/gitlab/embedded/bin/rake:22:in `load'", "/opt/gitlab/embedded/bin/rake:22:in `<top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `load'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in `kernel_load'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:27:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:332:in `exec'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:20:in `dispatch'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:11:in `start'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:34:in `block in <top (required)>'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/friendly_errors.rb100in `with_friendly_errors'", "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:26:in `<top (required)>'", "/opt/gitlab/embedded/bin/bundle:22:in `load'", "/opt/gitlab/embedded/bin/bundle:22:in `<main>'"]
Indexing wiki of IT_Adopt / IT_Adopt...
Done!
Indexing Project records... done
Indexing Issue records... done
Indexing MergeRequest records... done
Indexing Snippet records... done
Indexing Note records... done
Indexing Milestone records... done
root@fmgitlabapp02d:/etc/gitlab# cd /tmp
cc// @vsizov @nick.thomas