Skip to content

Add gitlab-ctl generate-secrets command

Andrew Patterson requested to merge 7701-gitlab-ctl-secret-generation-command into master

What does this MR do?

Add the generate-secrets command to gitlab-ctl. The -f|--file option is used to send the output to a specified file instead of the default /etc/gitlab/gitlab-secrets.json.

Related issues

Closes #7701 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.

Closes #7701 (closed)

Design

Modify SecretsHelper::write_to_gitlab_secrets

The existing SecretsHelper::write_to_gitlab_secrets class method will get the secrets from /etc/gitlab/gitlab.rb file and generate the /etc/gitlab/gitlab-secrets.json. We can modify this method to take optional path parameter which is then used to specify the output path for the secrets. If no path is not specified, output goes to /etc/gitlab/gitlab-secrets.json so backward compatibility is maintained.

Add the generate-secrets command to gitlab-ctl

Add the generate-secrets to the gitlab-ctl command with the optional argument -f|--file which can be used so specify the output file. If not specified, output goes to /etc/gitlab/gitlab-secrets.json. Help output is:

sudo gitlab-ctl generate-secrets --help 
Usage: omnibus-ctl [options]
    -f, --file=FILE                  Output secrets to file (default '/etc/gitlab/gitlab-secrets.json')

Calling SecretsHelper::write_to_gitlab_secrets

Calling the SecretsHelper::write_to_gitlab_secrets can be done in one of two ways. Either call it directly from within gitlab-ctl or indirectly by running a chef recipe that calls the SecretsHelper::write_to_gitlab_secrets method. Both have advantages and disadvantages:

Calling SecretsHelper::write_to_gitlab_secrets directly

This is simple, we just make the call inside the gitlab-ctl code that creates the generate-secrets command. However, the SecretsHelper::write_to_gitlab_secrets method calls the SecretsHelper::gather_gitlab_secrets which creates a map of which secrets to output using references to the Gitlab object which only seems to be available when running recipes. We could duplicate this code to use the node object, but that would make maintenance difficult as you any change would have to made in both maps. There may also be some sort of conversion routine that allows one to go back and forth betweend Gitlab and node.

Calling SecretsHelper::write_to_gitlab_secrets indirectly with a recipe

We can write a solo recipe that the gitlab-ctl which in turn calls the SecretsHelper::write_to_gitlab_secrets using the existing Gitlab by shelling out to the cinc-client command. This would work well and avoid the DRY issues found when calling directly. However, one of the requirements for this issue is the option to send output to stdout. The cinc-client is very noisy and has no quiet option, so status output would be mixed with desired secret JSON output. One possible work-around would be to patch the cinc-client to add a -q|--quiet option.

Resolution

Go with calling a generate_secrets recipe. Do not support stdout in this MR but leave for a later iteration.

Test plan

  • Install GitLab EE on a fresh system. Do not run `gitlab-ctl reconfigure or use any install option that would cause a reconfigure.

  • Run sudo gitlab-ctl --help and check for typos and English in generate-secrets entry.

  • Run sudo gitlab-ctl generate-secrets --help and check for typos and English.

  • Run sudo gitlab-ctl generate-secrets. You should get an error message about using the required -f|--file option.

  • Edit gitlab.rb and set package['generate_secrets_json_file''] = false.

  • Run sudo gitlab-ctl reconfigure.

    • No /etc/gitlab/gitlab-secrets.json` file should be created.

  • Edit gitlab.rb and set package['generate_secrets_json_file''] = true.

  • Run sudo gitlab-ctl generate-secrets -f /tmp/my-secrets.json. The command should exit after issue an error message about the package['generate_secrets_json_file''] setting.

  • Edit gitlab.rb and set package['generate_secrets_json_file''] = false.

  • Run sudo gitlab-ctl generate-secrets -f /xyzzy/my-secrets.json (non-existent directory). No error should be reported and no file should be created.

  • Run sudo gitlab-ctl generate-secrets -f /tmp/my-secrets.json.

    • Check /tmp/my-secrets.json for secrets.
    • Check permissions on owner on file (root:root 0600)
    • Check that EE-only secrets are generated, e.g., suggested_reviewers.
    • Copy secrets file (`sudo cp -p /tmp/my-secrets.json /tmp/my-secrets.json.orig')

  • Run `sudo gitlab-ctl generate-secrets -f /tmp/my-secrets.json'.

    • Compare secrets files (sudo diff /tmp/my-secrets.json.orig /tmp/my-secrets.json). They should be identical.

  • Edit gitlab.rb and set a secret, e.g, gitlab_pages['gitlab_secret'].

    • Run sudo gitlab-ctl generate-secrets -f /tmp/my-secrets.json.
    • Compare secrets files (sudo diff /tmp/my-secrets.json.orig /tmp/my-secrets.json). They should be identical expect that the edited secret should have the new value*.

  • Edit gitlab.rb and set package['generate_secrets_json_file''] = true.

  • Run sudo gitlab-ctl reconfigure.

    • Check /etc/gitlab/gitlab-secrets.json for secrets. It should contain the secret added above.

  • Install GitLab-EE on a fresh system.

  • Run sudo gitlab-ctl reconfigure. Check /etc/gitlab/gitlab-secrets.json for secrets.

  • Install GitLab CE on a fresh system. Do not run `gitlab-ctl reconfigure or use any install option that would cause a reconfigure.

    • Edit gitlab.rb and set package['generate_secrets_json_file''] = false.
    • Run sudo gitlab-ctl generate-secrets -f /tmp/my-secrets.json.
    • Check /tmp/my-secrets.json for secrets.
    • EE secrets, e.g., suggested_reviewers should not be set.
    • Edit gitlab.rb and set package['generate_secrets_json_file''] = true.
    • Run gitlab-ctl reconfigure
    • Check /etc/gitlab/gitlab-secrets.json for secrets.
Edited by Andrew Patterson

Merge request reports