Skip to content

Sidekiq jobs related with ElasticSearch are using "notes" index

Summary

Some Sidekiq jobs related with ElasticSearch are dying because we expect to only use one index: gitlab-production.

11 minutes ago 	elastic_indexer ElasticIndexerWorker "index", "DiscussionNote", 1585787
Elasticsearch::Transport::Transport::Errors::Unauthorized: [401] {"error":{"root_cause":[{"type":"status_exception","reason":"Sorry, your request is forbidden by policy.","header":{"WWW-Authenticate":"...

This behaviour does not happen with all the notes that are being created, this only happens with DiscussionNote.

Steps to reproduce

  • Enable ElasticSearch Advanced Global search on app settings and enable indexing.
  • Have restrictions on all the index names except gitlab-production, which is the default one.
  • Create DiscussionNotes and from time to time some of them will produce this dead jobs.

UPDATE: Create a discussion on either an Issue or Merge Request without referencing snippets or commits. Just write something and select "Start a discussion" instead of "Comment". This causes ALL indexing jobs for the entity to die because they are trying to use the notes index, which we restricted before.

Results of GitLab environment info

Expand for output related to GitLab environment info

System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown

GitLab information Version: 10.5.7-ee Revision: e6f48f0 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql DB Version: 9.6.2 URL: XYZ HTTP Clone URL: XYZ/some-group/some-project.git SSH Clone URL: ssh://git@XYZ:7999/some-group/some-project.git Elasticsearch: yes Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers: saml, kerberos_spnego

GitLab Shell Version: 6.0.3 Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories
  • secondary: /var/opt/gitlab/git-data2/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

UPDATE: It looks like the indexer is trying to use another index (called notes) apart from the main one called gitlab-production.

This looks like a bug, as this only happens with "DiscussionNote" entities, and more specifically, with discussions started without referencing a snippet or a commit.

Edited by Coung Ngo