Change gitlab:elastic rake to use a custom stdout logger

What does this MR do and why?

Related to #421298 (closed)

Part 1 of 3 MRs to change the gitlab:elastic:*** rake tasks into a service class and async worker

Description	Merge Request
⭐ Change gitlab:elastic rake tasks to use a custom stdout logger	!153855 (merged)
Move gitlab:elastic rake code into RakeTaskExecutorService	!154484 (merged)
Fully move gitlab:elastic:index to an async worker	!154610 (merged)

What this does

• cleanup some rubocop todos
• rename tasks in rake code to tasks_executor_service
• use a custom logger that only outputs to stdout. this was needed to allow a switch to using service code (and eventually an async worker for some tasks) in future MR
• spec updates to switch expectations from monitoring std_out to stub logger

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

gitlab:elastic:index

Click to expand with `elastic_index_use_trigger_indexing` FF enabled

➜ be rake gitlab:elastic:index
Index/alias 'gitlab-development' has been deleted
Index 'gitlab-development-issues-20240522-1528' with alias 'gitlab-development-issues' has been deleted
Index 'gitlab-development-notes-20240522-1528' with alias 'gitlab-development-notes' has been deleted
Index 'gitlab-development-merge_requests-20240522-1528' with alias 'gitlab-development-merge_requests' has been deleted
Index 'gitlab-development-commits-20240522-1528' with alias 'gitlab-development-commits' has been deleted
Index 'gitlab-development-epics-20240522-1528' with alias 'gitlab-development-epics' has been deleted
Index 'gitlab-development-users-20240522-1528' with alias 'gitlab-development-users' has been deleted
Index 'gitlab-development-wikis-20240522-1528' with alias 'gitlab-development-wikis' has been deleted
Index 'gitlab-development-projects-20240522-1528' with alias 'gitlab-development-projects' has been deleted
Index/alias 'gitlab-development-migrations' has been deleted
Index 'gitlab-development-issues-20240529-0029' has been created.
Alias 'gitlab-development-issues' -> 'gitlab-development-issues-20240529-0029' has been created.
Index 'gitlab-development-notes-20240529-0029' has been created.
Alias 'gitlab-development-notes' -> 'gitlab-development-notes-20240529-0029' has been created.
Index 'gitlab-development-merge_requests-20240529-0029' has been created.
Alias 'gitlab-development-merge_requests' -> 'gitlab-development-merge_requests-20240529-0029' has been created.
Index 'gitlab-development-commits-20240529-0030' has been created.
Alias 'gitlab-development-commits' -> 'gitlab-development-commits-20240529-0030' has been created.
Index 'gitlab-development-epics-20240529-0030' has been created.
Alias 'gitlab-development-epics' -> 'gitlab-development-epics-20240529-0030' has been created.
Index 'gitlab-development-users-20240529-0030' has been created.
Alias 'gitlab-development-users' -> 'gitlab-development-users-20240529-0030' has been created.
Index 'gitlab-development-wikis-20240529-0030' has been created.
Alias 'gitlab-development-wikis' -> 'gitlab-development-wikis-20240529-0030' has been created.
Index 'gitlab-development-projects-20240529-0030' has been created.
Alias 'gitlab-development-projects' -> 'gitlab-development-projects-20240529-0030' has been created.
Index '{"gitlab-development-20240529-0029"=>"gitlab-development"}' has been created.
Alias 'gitlab-development' → '{"gitlab-development-20240529-0029"=>"gitlab-development"}' has been created
Index status has been reset
Scheduling indexing with TriggerIndexingWorker
Scheduling indexing with TriggerIndexingWorker... done

Click to expand with `elastic_index_use_trigger_indexing` FF disabled

 echo "Feature.disable(:elastic_index_use_trigger_indexing)" | gdk rails c

➜ be rake gitlab:elastic:index
Index/alias 'gitlab-development' has been deleted
Index 'gitlab-development-issues-20240529-0029' with alias 'gitlab-development-issues' has been deleted
Index 'gitlab-development-notes-20240529-0029' with alias 'gitlab-development-notes' has been deleted
Index 'gitlab-development-merge_requests-20240529-0029' with alias 'gitlab-development-merge_requests' has been deleted
Index 'gitlab-development-commits-20240529-0030' with alias 'gitlab-development-commits' has been deleted
Index 'gitlab-development-epics-20240529-0030' with alias 'gitlab-development-epics' has been deleted
Index 'gitlab-development-users-20240529-0030' with alias 'gitlab-development-users' has been deleted
Index 'gitlab-development-wikis-20240529-0030' with alias 'gitlab-development-wikis' has been deleted
Index 'gitlab-development-projects-20240529-0030' with alias 'gitlab-development-projects' has been deleted
Index/alias 'gitlab-development-migrations' has been deleted
Index 'gitlab-development-issues-20240529-0051' has been created.
Alias 'gitlab-development-issues' -> 'gitlab-development-issues-20240529-0051' has been created.
Index 'gitlab-development-notes-20240529-0051' has been created.
Alias 'gitlab-development-notes' -> 'gitlab-development-notes-20240529-0051' has been created.
Index 'gitlab-development-merge_requests-20240529-0051' has been created.
Alias 'gitlab-development-merge_requests' -> 'gitlab-development-merge_requests-20240529-0051' has been created.
Index 'gitlab-development-commits-20240529-0051' has been created.
Alias 'gitlab-development-commits' -> 'gitlab-development-commits-20240529-0051' has been created.
Index 'gitlab-development-epics-20240529-0051' has been created.
Alias 'gitlab-development-epics' -> 'gitlab-development-epics-20240529-0051' has been created.
Index 'gitlab-development-users-20240529-0051' has been created.
Alias 'gitlab-development-users' -> 'gitlab-development-users-20240529-0051' has been created.
Index 'gitlab-development-wikis-20240529-0051' has been created.
Alias 'gitlab-development-wikis' -> 'gitlab-development-wikis-20240529-0051' has been created.
Index 'gitlab-development-projects-20240529-0051' has been created.
Alias 'gitlab-development-projects' -> 'gitlab-development-projects-20240529-0051' has been created.
Index '{"gitlab-development-20240529-0051"=>"gitlab-development"}' has been created.
Alias 'gitlab-development' → '{"gitlab-development-20240529-0051"=>"gitlab-development"}' has been created
Index status has been reset
Enqueuing Group level entities…
Indexing epics...
Indexing epics... done
Indexing group wikis...
Indexing group wikis... done
Enqueuing projects…. ✔ (56)
Indexing snippets...
Indexing snippets... done
Indexing users...
Indexing users... done

gitlab:elastic:index_group_entities

➜ be rake gitlab:elastic:index_group_entities
Enqueuing Group level entities…
Indexing epics...
Indexing epics... done
Indexing group wikis...
Indexing group wikis... done

disable_search_with_elasticsearch

➜ be rake gitlab:elastic:disable_search_with_elasticsearch
Setting `elasticsearch_search` has been disabled.

enable_search_with_elasticsearch

➜ be rake gitlab:elastic:enable_search_with_elasticsearch
Setting `elasticsearch_search` has been enabled.

gitlab:elastic:index_projects

➜ be rake gitlab:elastic:index_projects
Enqueuing projects…. ✔ (56)

gitlab:elastic:index_projects_status

➜ be rake gitlab:elastic:index_projects_status
Indexing is 100.00% complete (56/56 projects)

gitlab:elastic:index_snippets

➜ be rake gitlab:elastic:index_snippets
Indexing snippets...
Indexing snippets... done

gitlab:elastic:index_users

➜ be rake gitlab:elastic:index_users
Indexing users...
Indexing users... done

gitlab:elastic:index_epics

➜ be rake gitlab:elastic:index_epics
Indexing epics...
Indexing epics... done

gitlab:elastic:index_group_wikis

➜ be rake gitlab:elastic:index_group_wikis
Indexing group wikis...
Indexing group wikis... done

gitlab:elastic:delete_index

❯ be rake gitlab:elastic:delete_index
Index/alias 'gitlab-development' has been deleted
Index 'gitlab-development-issues-20240529-0051' with alias 'gitlab-development-issues' has been deleted
Index 'gitlab-development-notes-20240529-0051' with alias 'gitlab-development-notes' has been deleted
Index 'gitlab-development-merge_requests-20240529-0051' with alias 'gitlab-development-merge_requests' has been deleted
Index 'gitlab-development-commits-20240529-0051' with alias 'gitlab-development-commits' has been deleted
Index 'gitlab-development-epics-20240529-0051' with alias 'gitlab-development-epics' has been deleted
Index 'gitlab-development-users-20240529-0051' with alias 'gitlab-development-users' has been deleted
Index 'gitlab-development-wikis-20240529-0051' with alias 'gitlab-development-wikis' has been deleted
Index 'gitlab-development-projects-20240529-0051' with alias 'gitlab-development-projects' has been deleted
Index/alias 'gitlab-development-migrations' has been deleted

gitlab:elastic:create_empty_index

➜ be rake gitlab:elastic:create_empty_index
Index 'gitlab-development-issues-20240529-0112' has been created.
Alias 'gitlab-development-issues' -> 'gitlab-development-issues-20240529-0112' has been created.
Index 'gitlab-development-notes-20240529-0112' has been created.
Alias 'gitlab-development-notes' -> 'gitlab-development-notes-20240529-0112' has been created.
Index 'gitlab-development-merge_requests-20240529-0112' has been created.
Alias 'gitlab-development-merge_requests' -> 'gitlab-development-merge_requests-20240529-0112' has been created.
Index 'gitlab-development-commits-20240529-0112' has been created.
Alias 'gitlab-development-commits' -> 'gitlab-development-commits-20240529-0112' has been created.
Index 'gitlab-development-epics-20240529-0112' has been created.
Alias 'gitlab-development-epics' -> 'gitlab-development-epics-20240529-0112' has been created.
Index 'gitlab-development-users-20240529-0112' has been created.
Alias 'gitlab-development-users' -> 'gitlab-development-users-20240529-0112' has been created.
Index 'gitlab-development-wikis-20240529-0112' has been created.
Alias 'gitlab-development-wikis' -> 'gitlab-development-wikis-20240529-0112' has been created.
Index 'gitlab-development-projects-20240529-0112' has been created.
Alias 'gitlab-development-projects' -> 'gitlab-development-projects-20240529-0112' has been created.
Index '{"gitlab-development-20240529-0112"=>"gitlab-development"}' has been created.
Alias 'gitlab-development' → '{"gitlab-development-20240529-0112"=>"gitlab-development"}' has been created

gitlab:elastic:recreate_index

➜ be rake gitlab:elastic:recreate_index
Index/alias 'gitlab-development' has been deleted
Index 'gitlab-development-issues-20240529-0112' with alias 'gitlab-development-issues' has been deleted
Index 'gitlab-development-notes-20240529-0112' with alias 'gitlab-development-notes' has been deleted
Index 'gitlab-development-merge_requests-20240529-0112' with alias 'gitlab-development-merge_requests' has been deleted
Index 'gitlab-development-commits-20240529-0112' with alias 'gitlab-development-commits' has been deleted
Index 'gitlab-development-epics-20240529-0112' with alias 'gitlab-development-epics' has been deleted
Index 'gitlab-development-users-20240529-0112' with alias 'gitlab-development-users' has been deleted
Index 'gitlab-development-wikis-20240529-0112' with alias 'gitlab-development-wikis' has been deleted
Index 'gitlab-development-projects-20240529-0112' with alias 'gitlab-development-projects' has been deleted
Index/alias 'gitlab-development-migrations' has been deleted
Index 'gitlab-development-issues-20240529-0113' has been created.
Alias 'gitlab-development-issues' -> 'gitlab-development-issues-20240529-0113' has been created.
Index 'gitlab-development-notes-20240529-0113' has been created.
Alias 'gitlab-development-notes' -> 'gitlab-development-notes-20240529-0113' has been created.
Index 'gitlab-development-merge_requests-20240529-0113' has been created.
Alias 'gitlab-development-merge_requests' -> 'gitlab-development-merge_requests-20240529-0113' has been created.
Index 'gitlab-development-commits-20240529-0113' has been created.
Alias 'gitlab-development-commits' -> 'gitlab-development-commits-20240529-0113' has been created.
Index 'gitlab-development-epics-20240529-0113' has been created.
Alias 'gitlab-development-epics' -> 'gitlab-development-epics-20240529-0113' has been created.
Index 'gitlab-development-users-20240529-0113' has been created.
Alias 'gitlab-development-users' -> 'gitlab-development-users-20240529-0113' has been created.
Index 'gitlab-development-wikis-20240529-0113' has been created.
Alias 'gitlab-development-wikis' -> 'gitlab-development-wikis-20240529-0113' has been created.
Index 'gitlab-development-projects-20240529-0113' has been created.
Alias 'gitlab-development-projects' -> 'gitlab-development-projects-20240529-0113' has been created.
Index '{"gitlab-development-20240529-0113"=>"gitlab-development"}' has been created.
Alias 'gitlab-development' → '{"gitlab-development-20240529-0113"=>"gitlab-development"}' has been created

gitlab:elastic:reindex_cluster

➜ be rake gitlab:elastic:reindex_cluster
Reindexing job was successfully scheduled

gitlab:elastic:clear_index_status

➜ be rake gitlab:elastic:clear_index_status
Index status has been reset

gitlab:elastic:projects_not_indexed

➜ be rake gitlab:elastic:projects_not_indexed
Project 'a-zoekt-indexed-group/a-small-project' (ID: 39) isn't indexed.
Project 'namespace5/project-5' (ID: 55) isn't indexed.
Project 'namespace6/project-6' (ID: 56) isn't indexed.
Project 'flightjs/subgroup-b/project-a' (ID: 41) isn't indexed.
Project 'a-zoekt-indexed-group/gitlab-hq' (ID: 37) isn't indexed.
Project 'zoekt-indexing/i-have-many-branches' (ID: 27) isn't indexed.
Project 'root/a-user-project' (ID: 40) isn't indexed.
Project 'toolbox/gitlab-smoke-tests' (ID: 1) isn't indexed.
Project 'a-zoekt-indexed-group/Flight' (ID: 46) isn't indexed.
Project 'a-zoekt-indexed-group/archived-project' (ID: 48) isn't indexed.
Project 'toolbox/add-me-to-the-index' (ID: 31) isn't indexed.
Project 'gitlab-org/i-am-already-indexed' (ID: 32) isn't indexed.
Project 'submodule-group-test/project-with-submodule' (ID: 29) isn't indexed.
Project 'permissions-test/project-with-a-binary-file' (ID: 28) isn't indexed.
Project 'gitlab-org/gitlab-test' (ID: 2) isn't indexed.
Project 'gnuwget/Wget2' (ID: 4) isn't indexed.
Project 'gitlab-org/gitlab-shell' (ID: 3) isn't indexed.
Project 'permissions-test/public-project-repository-disabled' (ID: 26) isn't indexed.
Project 'a-not-indexed-group/a-not-indexed-project' (ID: 36) isn't indexed.
Project 'namespace7/project-7' (ID: 57) isn't indexed.
Project 'Commit451/a-256-sha-project' (ID: 43) isn't indexed.
Project 'zoekt-indexing/airflow-repo-template' (ID: 44) isn't indexed.
Project 'Commit451/test-no-repo' (ID: 45) isn't indexed.
Project 'a-zoekt-indexed-group/a-project-with-empty-repo' (ID: 47) isn't indexed.
Project 'submodule-group-test/i-am-a-submodule' (ID: 30) isn't indexed.
Project 'jashkenas/Underscore' (ID: 6) isn't indexed.
Project 'kai/gitlab-shell' (ID: 11) isn't indexed.
Project 'twitter/twitter-subgroup/Typeahead.Js' (ID: 8) isn't indexed.
Project 'Commit451/lab-coat' (ID: 5) isn't indexed.
Project 'francis.howe/gitlab-shell' (ID: 9) isn't indexed.
Project 'elke/gitlab-shell' (ID: 10) isn't indexed.
Project 'flightjs/Flight' (ID: 7) isn't indexed.
Project 'testing-elasticsearch-indexer/airflow-repo-template' (ID: 42) isn't indexed.
Project 'namespace1/project-1' (ID: 51) isn't indexed.
Project 'namespace2/project-2' (ID: 52) isn't indexed.
Project 'namespace3/project-3' (ID: 53) isn't indexed.
Project 'Commit451/another-group/regex-test' (ID: 50) isn't indexed.
Project 'namespace4/project-4' (ID: 54) isn't indexed.
Project 'a-zoekt-indexed-group/i-have-many-branches' (ID: 49) isn't indexed.
Project 'cedric/gitlab-shell' (ID: 12) isn't indexed.
Project 'hyo.ward/gitlab-shell' (ID: 13) isn't indexed.
Project 'christel/gitlab-shell' (ID: 14) isn't indexed.
Project 'Commit451/another-group/another-subgroup/my-transferred-project' (ID: 58) isn't indexed.
Project 'Commit451/another-group/another-subgroup/transfer-me' (ID: 59) isn't indexed.
Project 'gnuwget/test-transfer' (ID: 60) isn't indexed.
Project 'christel_littel/gitlab-shell' (ID: 15) isn't indexed.
Project 'davina.bartell/gitlab-shell' (ID: 16) isn't indexed.
Project 'frederick/gitlab-shell' (ID: 17) isn't indexed.
Project 'danial/gitlab-shell' (ID: 18) isn't indexed.
Project 'not-indexed-group/public-project' (ID: 19) isn't indexed.
Project 'permissions-test/private-project' (ID: 21) isn't indexed.
Project 'permissions-test/internal-project' (ID: 20) isn't indexed.
Project 'permissions-test/public-subgroup/public-project' (ID: 22) isn't indexed.
Project 'permissions-test/internal-subgroup/private-project' (ID: 24) isn't indexed.
Project 'permissions-test/internal-subgroup/internal-project' (ID: 23) isn't indexed.
Project 'permissions-test/private-subgroup/private-project' (ID: 25) isn't indexed.
56 out of 56 non-indexed projects shown.

gitlab:elastic:mark_reindex_failed

➜ be rake gitlab:elastic:mark_reindex_failed
Marked the current reindexing job as failed.

gitlab:elastic:list_pending_migrations

➜ be rake gitlab:elastic:list_pending_migrations
There are no pending migrations.

gitlab:elastic:estimate_cluster_size

➜ be rake gitlab:elastic:estimate_cluster_size
This GitLab instance combined repository and wiki size is 803.4 MiB.
By our estimates, your cluster size should be at least 401.7 MiB.
Please note that it is possible to index only selected namespaces/projects by using Advanced search indexing restrictions.

gitlab:elastic:estimate_shard_sizes

➜ be rake gitlab:elastic:estimate_shard_sizes
Using approximate counts to estimate shard counts for data indexed from database. This does not include repository data.
For single-node cluster recommendations, see http://gdk.test:3000/help/integration/advanced_search/elasticsearch#number-of-elasticsearch-shards
The approximate document counts, recommended shard size, and replica size for each index are:
- gitlab-development-issues:
   document count: 671
   recommended shards: 5
   recommended replicas: 1
- gitlab-development-notes:
   document count: 163
   recommended shards: 5
   recommended replicas: 1
- gitlab-development-merge_requests:
   document count: 107
   recommended shards: 5
   recommended replicas: 1
- gitlab-development-epics:
   document count: 40
   recommended shards: 5
   recommended replicas: 1
- gitlab-development-users:
   document count: 61
   recommended shards: 5
   recommended replicas: 1
- gitlab-development-projects:
   document count: 56
   recommended shards: 5
   recommended replicas: 1
Please note that it is possible to index only selected namespaces/projects by using Advanced search indexing restrictions. This estimate does not take into account indexing restrictions.

gitlab:elastic:pause_indexing

➜ be rake gitlab:elastic:pause_indexing
Pausing indexing...
Indexing is now paused.

gitlab:elastic:resume_indexing

➜ be rake gitlab:elastic:resume_indexing
Resuming indexing...
Indexing is now running.

gitlab:elastic:info

➜ be rake gitlab:elastic:info

Advanced Search
Server version:			8.11.1
Server distribution:		elasticsearch
Indexing enabled:		yes
Search enabled:			yes
Requeue Indexing workers:	yes
Pause indexing:			no
Indexing restrictions enabled:	no
File size limit:		1024 KiB
Indexing number of shards:	2
Max code indexing concurrency:	30

Indexing Queues
Initial queue:			0
Incremental queue:		0

Indices
- gitlab-development-20240529-0113:
	document_count: 2904
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-commits-20240529-0113:
	document_count: 103301
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-epics-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-issues-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-merge_requests-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-notes-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-projects-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-users-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-wikis-20240529-0113:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1
- gitlab-development-work_items-20240423-1216:
	document_count: 0
	number_of_shards: 5
	number_of_replicas: 1

How to set up and validate locally

run each rake task to test the output