Elasticsearch searching causes increased elastic_thread_pooling saturation

Summary

It appears that enabling advanced search for global scope by default may be causing increased elastic cluster thread pooling to become saturated.

Steps to reproduce

This issue is currently on-going: gitlab-com/gl-infra/production#2499 (closed)

Example Project

N/A

What is the current bug behavior?

(What actually happens)

What is the expected correct behavior?

(What you should see instead)

Relevant logs and/or screenshots

Elastic thread pools saturation graph: Screen_Shot_2020-08-07_at_6.51.05_PM

Web Puma Error Ratio: Screen_Shot_2020-08-07_at_6.52.09_PM

Output of checks

This bug happens on GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Revert this change possibly: !38093 (merged)