Clean up Gitaly feature flags - November 2022
When rolling out gitaly_always_log_feature_flags flag that attaches feature_flags field to Gitaly logs, I noticed that there are more than 60 flags enabled on Staging and Production.
In the codebase, there are just around a dozen of flags in use. The rest are redundant and should be removed from the database. Both GitLab Rails and Gitaly validate flag definition. If a flag's definition is removed from the code base, there is no reason to keep it. Apart from reducing the logging volume, deleting finished/abandoned flags reduces the mental load when we need to debug something caused by a flag's state. Sometimes, a flag is misleading! For example, gitaly_grpc_tree_entry_not_found is set to 25%. However, in fact, this flag was removed a long time ago.
The following flags are fetched from chatops. I wrote a small script for analyzing them. This script pulls out the historical git data to determine the point of introduction and point of deletion in the past. In the future, we can convert that script to a bot. It can automatically create monthly housekeeping issue. I ran against gitlab-org/gitlab and gitlab-org/gitaly repositories. Not sure there is any other place using the flag implicitly?
Removing flags is easy, but a bit time-consuming. Currently, chatops doesn't support mass flag deletion. But I'm a bit skeptical about supporting it. The only way now is to remove the flag one by one, or write an API script (need Admin Access Token) to accelerate the progress.
/chatops run feature set delete gitaly_apply_patch_respect_committer_timezone
/chatops run feature set delete gitaly_atomic_remove_repository
/chatops run feature set delete gitaly_cherry_pick_structured_errors
...Flag analysis
| Flag | Environment | Contact | Introduced in | Removed in | Safe to delete |
|---|---|---|---|---|---|
| gitaly_apply_patch_respect_committer_timezone | Staging, Production | Author: Patrick Steinhardt | gitaly 14.6.0-rc1 | gitaly 14.8.0-rc1 |
|
| gitaly_atomic_remove_repository | Staging, Production | Author: Patrick Steinhardt | gitaly 14.4.0-rc43 | gitaly 14.6.0-rc1 |
|
| gitaly_cherry_pick_structured_errors | Staging, Production | Author: John Cai | gitaly 15.1.0-rc1 | gitaly 15.2.0-rc1 |
|
| gitaly_command_stats_metrics | Staging, Production | Author: Igor Wiedler | gitaly 14.9.0-rc5 | gitaly 14.10.0-rc1 |
|
| gitaly_concurrency_queue_enforce_max | Staging, Production | ||||
| gitaly_concurrency_queue_max_wait | Staging, Production | ||||
| gitaly_convert_err_to_status | Staging, Production | Author: Pavlo Strokov | gitaly 15.6.0-rc2 | ||
| gitaly_create_repository_from_bundle_atomic_fetch | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc4 | gitaly 14.2.0-rc1 |
|
| gitaly_delete_refs_structured_errors | Staging, Production | Author: John Cai | gitaly 15.3.0-rc3 | ||
| gitaly_fetch_internal_no_alternate_refs | Staging, Production | Author: Patrick Steinhardt | gitaly 14.2.0-rc2 | gitaly 14.2.0-rc2 |
|
| gitaly_fetch_internal_with_sidechannel | Staging, Production | Author: Patrick Steinhardt | gitaly 14.8.0-rc1 | gitaly 14.8.0-rc1 |
|
| gitaly_fetch_into_object_pool_optimize_repository | Staging, Production | Author: Patrick Steinhardt | gitaly 15.2.0-rc1 | gitaly 15.2.0-rc1 |
|
| gitaly_fetch_into_object_pool_prune_refs | Staging, Production | Author: Patrick Steinhardt | gitaly 15.3.0-rc1 | gitaly 15.4.0-rc2 |
|
| gitaly_find_all_tags_pipeline | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc3 | gitaly 14.2.0-rc2 |
|
| gitaly_find_tag_structured_error | Staging, Production | Author: Patrick Steinhardt | gitaly 15.3.0-rc3 | gitaly 15.4.0-rc2 |
|
| gitaly_get_archive_lfs_filter_process | Staging, Production | Author: Patrick Steinhardt | gitaly 15.1.0-rc1 | gitaly 15.1.0-rc1 |
|
| gitaly_git2go_merge_gob | Staging, Production | Author: Aleksei Lipniagov | gitaly 14.3.0-rc2 | gitaly 14.3.0-rc2 |
|
| gitaly_git_v2351_with_fetch_speedups | Staging, Production | Author: Patrick Steinhardt | gitaly 14.9.0-rc1 | gitaly 14.9.0-rc5 |
|
| gitaly_git_v2371gl1 | Staging, Production | ||||
| gitaly_git_v238 | Staging, Production | Author: John Cai | gitaly 15.4.0-rc2 | ||
| gitaly_go_language_stats | Staging, Production | Author: Toon Claes | gitaly 15.1.0-rc1 | gitaly 15.6.0-rc2 |
|
| gitaly_go_set_config | Staging, Production | Author: Patrick Steinhardt | gitaly 14.2.0-rc2 | gitaly 14.3.0-rc1 |
|
| gitaly_go_user_apply_patch | Staging, Production | Author: Sami Hiltunen | gitaly 14.2.0-rc2 | gitaly 14.3.0-rc1 |
|
| gitaly_go_user_cherry_pick | Staging, Production | Author: Toon Claes | gitaly 13.11.0-rc1 | gitaly 13.11.0-rc1 |
|
| gitaly_go_user_revert | Staging, Production | Author: James Fargher | gitaly 13.12.0-rc1 | gitaly 13.12.0-rc1 |
|
| gitaly_go_user_update_branch | Staging, Production | Author: Ævar Arnfjörð Bjarmason | gitaly 13.9.0-rc1 | gitaly 13.12.0-rc1 |
|
| gitaly_go_user_update_submodule | Staging, Production | Author: Zeger-Jan van de Weg | gitaly 13.9.0-rc1 | gitaly 13.12.0-rc1 |
|
| gitaly_grpc_tree_entry_not_found | Staging, Production | Author: Mikhail Mazurskiy | gitaly 13.11.0-rc1 | gitaly 14.0.0-rc1 |
|
| gitaly_hooks_in_tempdir | Staging, Production | Author: Patrick Steinhardt | gitaly 14.6.0-rc1 | gitaly 14.8.0-rc1 |
|
| gitaly_lfs_pointers_pipeline | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc3 | gitaly 14.1.0-rc4 |
|
| gitaly_log_command_stats | Staging, Production | Author: Patrick Steinhardt | gitaly 13.11.0-rc1 | gitaly 13.12.0-rc1 |
|
| gitaly_maintenance_operation_routing | Staging, Production | Author: John Cai | gitaly 14.9.0-rc5 | gitaly 14.10.0-rc1 |
|
| gitaly_node_error_cancels_voter | Staging, Production | Author: Justin Tobler | gitaly 15.4.0-rc2 | ||
| gitaly_pack_objects_limiting_repo | Staging, Production | Author: John Cai | gitaly 15.6.0-rc2 | ||
| gitaly_pack_objects_metrics | Staging, Production | Author: John Cai | gitaly 15.1.0-rc1 | gitaly 15.3.0-rc3 |
|
| gitaly_quarantine | Staging, Production | Author: Patrick Steinhardt | gitaly 14.2.0-rc2 | gitaly 14.2.0-rc2 |
|
| gitaly_quarantined_resolve_conflicts | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc4 | gitaly 14.2.0-rc2 |
|
| gitaly_quarantined_user_create_tag | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc4 | gitaly 14.2.0-rc2 |
|
| gitaly_rename_repository_locking | Staging, Production | ||||
| gitaly_replicate_repository_direct_fetch | Staging, Production | Author: Patrick Steinhardt | gitaly 13.12.6 | gitaly 14.2.0-rc1 |
|
| gitaly_run_cmds_in_cgroup | Staging, Production | ||||
| gitaly_squash_using_merge | Staging, Production | Author: Piotr Stankowski | gitaly 15.0.0-rc1 | gitaly 15.0.0-rc2 |
|
| gitaly_transactional_symbolic_ref_updates | Staging, Production | Author: John Cai | rails 14.9.0-pre |
|
|
| gitaly_tree_entries_new_page_token_format | Staging, Production | Author: Vasilii Iakliushin | gitaly 15.4.0-rc2 | ||
| gitaly_tree_entries_via_ls_tree | Staging, Production | Author: Patrick Steinhardt | gitaly 14.4.0-rc43 | gitaly 14.4.0-rc43 |
|
| gitaly_tx_apply_gitattributes_two_phase_removal | Staging, Production | Author: Patrick Steinhardt | gitaly 14.6.0-rc1 | gitaly 14.6.0-rc1 |
|
| gitaly_tx_atomic_repository_creation | Staging, Production | Author: Patrick Steinhardt | gitaly 14.4.0-rc43 | gitaly 14.6.0-rc1 |
|
| gitaly_tx_config | Staging, Production | Author: Patrick Steinhardt | gitaly 14.0.0-rc2 | gitaly 14.0.0-rc2 |
|
| gitaly_tx_extended_file_locking | Staging, Production | Author: Patrick Steinhardt | gitaly 14.3.0-rc1 | gitaly 14.3.0-rc2 |
|
| gitaly_tx_file_locking | Staging, Production | Author: Patrick Steinhardt | gitaly 14.3.0-rc1 | gitaly 14.3.0-rc2 |
|
| gitaly_tx_remote | Staging, Production | Author: Patrick Steinhardt | gitaly 14.0.0-rc2 | gitaly 14.0.0-rc2 |
|
| gitaly_tx_remove_repository | Staging, Production | Author: Patrick Steinhardt | gitaly 14.0.5 | gitaly 14.2.0-rc2 |
|
| gitaly_tx_two_phase_delete_refs | Staging, Production | Author: Patrick Steinhardt | gitaly 14.6.0-rc1 | gitaly 14.6.0-rc1 |
|
| gitaly_updateref_verify_state_changes | Staging, Production | ||||
| gitaly_upload_pack_hide_refs | Staging, Production | Author: Patrick Steinhardt | gitaly 15.2.0-rc1 | gitaly 15.3.0-rc1 |
|
| gitaly_use_bundled_git | Staging, Production | Author: Patrick Steinhardt | gitaly 14.8.0-rc1 | gitaly 14.9.0-rc5 |
|
| gitaly_user_create_branch_structured_errors | Staging, Production | Author: Karthik Nayak | gitaly 15.4.0-rc1 | gitaly 15.4.0-rc2 |
|
| gitaly_user_create_tag_structured_errors | Staging, Production | Author: Patrick Steinhardt | gitaly 15.2.0-rc1 | gitaly 15.4.0-rc2 |
|
| gitaly_user_delete_branch_structured_errors | Staging, Production | Author: Patrick Steinhardt | gitaly 15.1.0-rc1 | gitaly 15.1.0-rc1 |
|
| gitaly_user_merge_branch_access_error | Staging, Production | Author: Patrick Steinhardt | gitaly 14.3.0-rc1 | gitaly 14.8.0-rc1 |
|
| gitaly_user_rebase_confirmable_improved_error_handling | Staging, Production | Author: John Cai | gitaly 14.9.0-rc5 | gitaly 15.2.0-rc1 |
|
| gitaly_user_squash_improved_error_handling | Staging, Production | Author: Patrick Steinhardt | gitaly 14.8.0-rc1 | gitaly 14.9.0-rc5 |
|
| gitaly_user_squash_quarantined_voting | Staging, Production | Author: Patrick Steinhardt | gitaly 14.9.0-rc5 | gitaly 15.0.0-rc1 |
|
| gitaly_user_squash_without_worktree | Staging, Production | Author: Patrick Steinhardt | gitaly 14.1.0-rc4 | gitaly 14.2.0-rc2 |
|
| repository_tree_gitaly_pagination | Staging | ||||
| gitaly_catfile_repo_size | Staging, Production | ||||
| gitaly_go_find_license | Staging, Production | Author: Kirill | gitaly 14.2.0-rc2 |
|
Edited by Quang-Minh Nguyen