Skip to content

Refactor Ansible for speed

Grant Young requested to merge gy-ansible-improvements into main

What does this MR do?

MR refactors Ansible with numerous improvements to streamline as follows:

  • Facts Gathering is now only run once
  • GitLab Rails Console calls have been greatly reduced on average. These take 30s and more to boot up so have been streamlined and unified notably for subsequent runs. Only one call is made to config check and if all config matches as expected no further calls are made.
  • common_prep tag added to allow for skipping common prep tasks if user is confident the prep is not required
  • postgres_external_prep tag added to allow skipping setup of external database if already run
  • Superfluous Secrets call removed for single playbook path
  • Superfluous Gitaly facts call removed for GitLab charts path

In addition to the above a new playbook for clearing Consul data (as workaround for omnibus-gitlab#8378) and the setting of the new gitlab_environment_toolkit_instance setting for service ping usage has been added.

On a Cloud Native Hybrid setup with external Postgres and Redis, running through all.yml for upgrade runs has been almost reduced by half:

Before (8 minutes, 45 seconds) 
Playbook run took 0 days, 0 hours, 8 minutes, 45 seconds

Friday 17 May 2024  14:29:16 +0100 (0:00:00.053)       0:08:45.036 ************
===============================================================================
post_configure -------------------------------------------------------- 155.10s
gitlab_charts --------------------------------------------------------- 135.08s
common ---------------------------------------------------------------- 110.46s
gather_facts ----------------------------------------------------------- 87.39s
gitaly ----------------------------------------------------------------- 33.25s
include_role ------------------------------------------------------------ 3.74s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
total ----------------------------------------------------------------- 525.02s

Friday 17 May 2024  14:29:16 +0100 (0:00:00.053)       0:08:45.035 ************
===============================================================================
gitlab_charts : Install GitLab Charts ---------------------------------------------------------------------------------------------------------------------------------------------- 83.72s
post_configure : Disable Write to "authorized_keys" file setting via GitLab Toolbox pod -------------------------------------------------------------------------------------------- 35.99s
post_configure : Check for existing license or configure new subscription / license via GitLab Toolbox pod ------------------------------------------------------------------------- 35.91s
post_configure : Get and save Environment Settings via GitLab Toolbox pod ---------------------------------------------------------------------------------------------------------- 33.84s
post_configure : Get and save Environment Settings via GitLab Toolbox pod ---------------------------------------------------------------------------------------------------------- 33.83s
gitaly : Reconfigure Gitaly -------------------------------------------------------------------------------------------------------------------------------------------------------- 12.80s
Gathering Facts -------------------------------------------------------------------------------------------------------------------------------------------------------------------- 11.61s
Gathering Facts -------------------------------------------------------------------------------------------------------------------------------------------------------------------- 10.27s
gitlab_charts : Gather Gitaly group facts ------------------------------------------------------------------------------------------------------------------------------------------- 9.41s
gitlab_charts : Enable required Postgres extensions --------------------------------------------------------------------------------------------------------------------------------- 8.30s
common : Create select GitLab dirs if missing --------------------------------------------------------------------------------------------------------------------------------------- 8.10s
common : Run apt update ------------------------------------------------------------------------------------------------------------------------------------------------------------- 7.18s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.71s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.17s
gitlab_charts : Configure GitLab Chart password secrets ----------------------------------------------------------------------------------------------------------------------------- 5.52s
post_configure : Check for Toolbox Deployment --------------------------------------------------------------------------------------------------------------------------------------- 5.28s
post_configure : Wait for Toolbox deployment to become ready ------------------------------------------------------------------------------------------------------------------------ 5.02s
common : Populate service facts ----------------------------------------------------------------------------------------------------------------------------------------------------- 4.92s
common : Populate package facts ----------------------------------------------------------------------------------------------------------------------------------------------------- 4.56s
common : Write GitLab Secrets file -------------------------------------------------------------------------------------------------------------------------------------------------- 4.56s
gitaly : Setup GitLab config file --------------------------------------------------------------------------------------------------------------------------------------------------- 4.54s
common : Write GitLab Secrets file -------------------------------------------------------------------------------------------------------------------------------------------------- 4.48s
common : Install system packages (Ubuntu / Debian) ---------------------------------------------------------------------------------------------------------------------------------- 4.38s
common : Run Automatic Security Upgrades directly (Ubuntu / Debian) ----------------------------------------------------------------------------------------------------------------- 4.37s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.30s
common : Install python packages ---------------------------------------------------------------------------------------------------------------------------------------------------- 4.09s
common : Unlock GitLab package installs --------------------------------------------------------------------------------------------------------------------------------------------- 3.95s
gitaly : Restart Non Gitaly components ---------------------------------------------------------------------------------------------------------------------------------------------- 3.86s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.84s
common : Lock GitLab package updates ------------------------------------------------------------------------------------------------------------------------------------------------ 3.84s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.80s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.77s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.76s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.74s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.74s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.74s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.74s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.73s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.72s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.67s
common : Install GitLab repo package (deb) ------------------------------------------------------------------------------------------------------------------------------------------ 3.65s
common : Install unattended-upgrades package ---------------------------------------------------------------------------------------------------------------------------------------- 3.62s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.58s
Gathering Facts --------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.50s
After (4 minutes, 49 seconds) 
Playbook run took 0 days, 0 hours, 4 minutes, 49 seconds

gitlab_charts : Install GitLab Charts ---------------------------------------------------------------------------------------------------------------------------------------------- 92.70s
post_configure : Get and save Environment Settings and License plan via GitLab Toolbox pod ----------------------------------------------------------------------------------------- 34.88s
gitaly : Reconfigure Gitaly -------------------------------------------------------------------------------------------------------------------------------------------------------- 12.63s
gitlab_charts : Enable required Postgres extensions --------------------------------------------------------------------------------------------------------------------------------- 7.57s
common : Gather facts (common) ------------------------------------------------------------------------------------------------------------------------------------------------------ 6.52s
common : Install system packages (Ubuntu / Debian) ---------------------------------------------------------------------------------------------------------------------------------- 5.63s
gitlab_charts : Configure GitLab Chart password secrets ----------------------------------------------------------------------------------------------------------------------------- 5.23s
common : Gather service facts (common) ---------------------------------------------------------------------------------------------------------------------------------------------- 4.87s
common : Run Automatic Security Upgrades directly (Ubuntu / Debian) ----------------------------------------------------------------------------------------------------------------- 4.40s
common : Gather package facts (common) ---------------------------------------------------------------------------------------------------------------------------------------------- 4.39s
common : Write GitLab Secrets file -------------------------------------------------------------------------------------------------------------------------------------------------- 4.29s
gitaly : Setup GitLab config file --------------------------------------------------------------------------------------------------------------------------------------------------- 4.26s
common : Write GitLab Secrets file -------------------------------------------------------------------------------------------------------------------------------------------------- 4.25s
common : Unlock GitLab package installs --------------------------------------------------------------------------------------------------------------------------------------------- 3.90s
common : Install python packages ---------------------------------------------------------------------------------------------------------------------------------------------------- 3.90s
common : Lock GitLab package updates ------------------------------------------------------------------------------------------------------------------------------------------------ 3.77s
gitaly : Restart Non Gitaly components ---------------------------------------------------------------------------------------------------------------------------------------------- 3.71s
common : Install GitLab repo package (deb) ------------------------------------------------------------------------------------------------------------------------------------------ 3.58s
common : Install unattended-upgrades package ---------------------------------------------------------------------------------------------------------------------------------------- 3.50s

ansible-playbook -i environments/3k_test_hybrid_aws_services/inventory     56.25s user 25.26s system 27% cpu 4:53.63 total

Related issues

Relates #875 (closed)

Author's checklist

When ready for review, the Author applies the workflowready for review label and mention @gl-quality/get-maintainers:

  • Merge request:
    • Corresponding Issue raised and reviewed by the GET maintainers team.
    • Merge Request Title and Description are up-to-date, accurate, and descriptive
    • MR targeting the appropriate branch
    • MR has a green pipeline
    • MR has no new security alerts in the widget from the Secret Detection and IaC Scan (SAST) jobs.
  • Code:
    • Check the area changed works as expected. Consider testing it in different environment sizes (1k,3k,10k,etc.).
    • Documentation created/updated in the same MR.
    • If this MR adds an optional configuration - check that all permutations continue to work.
    • For Terraform changes: set up a previous version environment, then run a terraform plan with your new changes and ensure nothing will be destroyed. If anything will be destroyed and this can't be avoided please add a comment to the current MR.
  • Create any follow-up issue(s) to support the new feature across other supported cloud providers or advanced configurations. Create 1 issue for each provider/configuration. Contact the Quality Enablement team if unsure.
Edited by Grant Young

Merge request reports